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ABSTRACT 


HOIUM,  DEBRA  KAY.  The  Severe  Weather  Warning  Process  using  the  WSR-88D  at 
the  Raleigh  Weather  Forecast  Office.  (Under  the  direction  of  Allen  J.  Riordan.) 

For  over  a  century,  meteorologists  have  been  attempting  to  improve  severe 
thunderstorm  and  tornado  forecasts.  In  recent  years,  the  Doppler  radar  was  developed  in 
an  attempt  to  increase  severe  weather  warning  timeliness  and  accuracy.  This  research  is 
a  preliminary  analysis  of  the  severe  weather  warning  process  using  the  WSR-88D  at  the 
Raleigh  Weather  Forecast  Office  (RDU  WFO). 

A  schematic  representation  of  the  warning  process  was  developed  based  on  68 
thoroughly  documented  warnings  during  1995.  A  flow-chart  has  been  developed  to 
describe  the  decision  making  process.  Most  warnings  have  an  initiator  followed  by  a 
trigger.  The  initiator  causes  the  radar  operator  to  consider  the  issuance  of  a  warning  and 
marks  a  period  of  intensive  cell  investigation.  A  final  event,  termed  the  trigger,  leads  to 
the  decision  to  issue  the  warning.  The  trigger  for  severe  thunderstorm  warnings  is  most 
frequently  a  reflectivity  based  Doppler  radar  product  while  for  tornado  warnings  Doppler 
velocity  products  are  most  commonly  utilized.  Ground  truth  reports  are  still  important  in 
the  Doppler  radar  era  especially  for  the  decisions  not  to  issue  warnings  and  quick 
decisions  to  issue  warnings,  called  immediate  trigger  warnings.  After  the  warning  is 
issued,  85  percent  of  the  warned  counties  are  called  leading  to  the  verification  of  38 
percent  of  the  warnings  during  the  severe  weather  episode.  Seventy  percent  of  these 
severe  weather  reports  are  accurate  if  Storm  Data  is  used  as  the  ground  truth. 


The  probability  of  detection  (POD)  at  the  RDU  WFO  has  statistically  significantly 
improved  since  operational  use  of  the  WSR-88D  began  in  March  of  1994.  However,  as 
of  November  1995  both  the  false  alarm  rate  (FAR)  and  critical  success  index  (CSI)  have 
gotten  worse  with  the  use  of  the  Doppler  radar.  It  has  further  been  found  that  the  FAR 
has  increased  from  1994  to  1995.  The  hypothesis  for  such  increases  in  the  FAR  is  that 
there  is  a  tendency  to  over  warn  until  the  forecaster  gains  operational  experience  on  the 
WSR-88D.  More  forecasters  were  having  their  first  operational  experiences  with  the 
Doppler  radar  during  1995  than  1994.  Placing  the  warnings  in  categories  revealed  that 
the  POD  is  higher  during  periods  with  severe  thunderstorm  or  tornado  watches  as  well  as 
for  severe  weather  episodes  with  more  than  four  warnings.  Finally,  the  population 
density  and  per  capita  income  do  not  have  a  statistically  significant  impact  on  either  the 
verification  rate  or  the  number  of  warnings  issued. 
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1.  INTRODUCTION 


“In  a  typical  year,  the  United  States  can  expect  a  staggering  assault  by  the 
elements;  some  10,000  violent  thunderstorms,  500  floods,  1000  tornadoes,  and  several 
hurricanes”  (Friday,  1994).  Clearly,  this  situation  emphasizes  the  importance  of  timely 
and  accurate  severe  weather  warnings  to  protect  the  American  public.  However,  the  first 
weather  warnings  were  issued  only  a  century  and  a  half  ago. 

1.1  The  First  Severe  Weather  Warnings 

Professor  Joseph  Henry,  secretary  of  the  Smithsonian  Institution,  devised  the  first 
weather  warning  system  in  the  United  States  in  1849  using  a  telegraph  network  and  150 
volunteers  (Hughes,  1970).  His  system  grew  to  include  over  500  stations  by  1860,  but 
the  Civil  War  caused  reductions  thereafter  (Whitnah,  1961). 

The  modem  severe  thunderstorm  and  tornado  watch  procedures  were  developed 
with  the  advent  of  the  National  Severe  Storms  Forecast  Center  (NSSFC)  in  1953  (The 
Tornado  Project,  1994).  Possibly  as  a  result,  1953  was  the  last  year  that  a  single  tornado 
killed  over  100  people  (The  Tornado  Project,  1994).  Severe  weather  warnings  were 
developed  soon  after  the  first  watches.  While  both  watch  and  warning  procedures  have 
been  refined,  the  basic  system  is  still  used  today  and  will  be  presented  in  the  following 
section. 

1.2  Verification  Techniques 

From  the  earliest  forecasts,  meteorologists  have  been  interested  in  assessing  their 
performance.  John  P.  Finley  devised  the  first  verification  scheme  for  his  tornado 


predictions  in  the  1880’s  (Flueck,  1987).  Finley  expressed  a  view  which  is  still 
considered  valid  today,  “...it  requires  as  much,  and  often  more,  study  to  say  that  no 
tornadoes  will  occur  as  to  make  the  prediction  that  conditions  are  favorable  for  their 
development”  (Galway,  1985  quoting  Finley,  1884).  To  incorporate  accurate  forecasts 
for  non-tomadic  events,  Finley  considered  a  verified  forecast  as  any  correctly  identified 
tornado  or  no  tornado  period  (Flueck,  1987).  Using  this  method,  Finley’s  verification 
scores  ranged  from  96  to  99  percent  (Flueck,  1987).  However,  if  Finley  had  never 
predicted  a  tornado,  his  verification  rate  would  have  been  98  percent  (Flueck,  1987). 
Clearly,  there  was  a  problem  with  Finley’s  verification  scheme  and  the  debate  for  the 
proper  method  of  analyzing  forecast  skill  and  accuracy  continues  today. 

Current  severe  weather  warning  and  verification  procedures  are  defined  by  the 
National  Severe  Storms  Forecast  Center  (NSSFC),  now  called  the  Storm  Prediction 
Center  (SPC),  in  Kansas  City,  MO  (Crowther  and  Halmstad,  1994)  and  will  be  used  in 
this  report.  The  NSSFC’s  definition  of  a  “severe  local  storm  event”  is  any  tornado  or 
wind  gust  in  excess  of  50  knots  or  thunderstorm  wind  damage  or  hail  with  a  diameter  3/4 
inch  or  greater  (Grenier  and  Halmstad,  1986).  Multiple  reports  of  severe  local  storm 
events  that  occur  within  the  same  county  are  recorded  as  one  event  if  the  reports  are 
within  ten  square  miles  and  fifteen  minutes  of  each  other  (Crowther  and  Halmstad, 
1995).  As  an  exception  to  this  rule,  the  following  cases  are  always  treated  as  individual 
severe  local  storm  events: 

1)  all  distinct  tornadoes 

2)  wind  gusts  in  excess  of  65  knots  and  hail  with  a  diameter  greater  than  two  inches,  and 
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3)  all  events  with  reported  fatalities,  injuries,  or  more  than  half  a  million  dollars  worth  of 
damage  (Crowther  and  Halmstad,  1995). 

In  this  report,  severe  local  storm  events  will  be  referred  to  as  severe  weather  events  or 
simply  events. 

Severe  thunderstorm  and  tornado  watches  are  issued  by  the  NSSFC  for  areas 
where  severe  local  storm  events  are  forecast  to  occur  (Crowther  and  Halmstad,  1993). 
These  watches  usually  include  areas  of  roughly  50,000  square  kilometers  and  are 
typically  valid  for  up  to  six  hours. 

In  contrast,  the  local  Weather  Forecast  Offices  (WFO)  of  the  National  Weather 
Service  (NWS)  issue  warnings  to  “alert  the  public  to  an  imminent  or  existing  severe 
thunderstorm  or  tornado”  (Crowther  and  Halmstad,  1995).  Severe  thunderstorm  and 
tornado  warnings  are  issued  for  all  or  part  of  a  county  and  are  generally  valid  for  less 
than  one  hour.  Each  WFO  issues  warning  to  their  defined  area  of  responsibility. 
Appendix  7. 1  shows  the  Raleigh  Warning  Area  (RWA). 

The  local  WFO  then  uses  phone  calls,  ham  radio  reports,  damage  surveys  and 
newspaper  articles  to  compile  a  list  of  all  events  in  their  region.  The  reports  of  all 
WFO’s  are  compiled  in  Storm  Data:  A  Composite  of  Outstanding  Storms  which  then 
becomes  the  official  verification  source.  This  research  uses  the  July  and  August,  1994 
editions  of  Storm  Data  as  well  as  press-ready  versions  of  the  North  Carolina  entries  for 
June  of  1994  and  all  of  1995  (Lemons,  1995). 

A  warning  verifies  if  there  is  a  report  in  Storm  Data  of  an  event  within  the  warned 
county  and  during  the  valid  warning  period  (Crowther  and  Halmstad,  1995).  Any  severe 
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local  storm  event  verifies  either  a  severe  thunderstorm  or  a  tornado  warning  (Crowther 
and  Halmstad,  1995).  This  is  clearly  a  generous  verification  procedure  since  for 
successful  warnings,  the  type  of  warning  and  type  of  severe  event  do  not  have  to  match. 
For  example,  a  wind  gust  in  excess  of  50  knots  verifies  a  tornado  warning.  Since 
warnings  are  issued  by  county,  verification  is  also  done  by  county.  This  means  that  if  a 
warning  is  issued  for  two  counties,  each  warned  county  is  treated  individually  for 
verification  purposes  (Crowther  and  Halmstad,  1995). 

The  verification  results  are  then  used  to  compute  the  verification  statistics.  Three 
of  the  most  common  verification  statistics  used  today  are  the  probability  of  detection 
(POD),  the  false  alarm  rate  (FAR)  and  the  critical  success  index  (CSI).  First,  there  are 
four  basic  variables  involved  in  verification  statistics  as  shown  in  Table  1.1. 


Table  1.1.  Contingency  table  showing  variables  involved  in  verification  statistics 
(Schaefer,  1990). 

FORECASTS 


EVENTS 


Yes 

No 

Yes 

X 

Y 

No 

Z 

W 

The  number  of  correct  forecasts  for  severe  local  storm  events  and  non-severe  local  storm 
events  are  X  and  W  respectively,  while  the  total  number  of  unverified  positive  and 
negative  forecasts  are  Z  and  Y.  While  researchers  agree  that  W  should  not  be  ignored, 
there  is  not  a  standard  way  to  deal  with  the  infinite  number  of  correct  decisions  not  to 
warn  a  county.  In  the  remainder  of  this  report,  “misses”  will  be  defined  as  Y  and 
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“episodes”  will  refer  to  the  time  periods  when  warnings  are  issued  with  a  maximum  of 
three  hours  between  warnings.  Notice  also  that  the  number  of  events  is  X  +  Y. 

Now  the  verification  statistics  can  be  defined.  The  probability  of  detection  is 
defined  as: 

POD  =  (Schaefer,  1 990) 

X  +  Y 

and  is  simply  the  percentage  of  events  for  which  a  warning  has  been  issued  (Schaefer, 
1990).  A  perfect  POD  is  1.00.  The  false  alarm  rate  is  a  measure  of  the  failure  to  exclude 
non-severe  events  and  is  defined  as: 

FAR  =  — ^  (Schaefer,  1990) 

X  4"  Z 

and  should  be  minimized.  Both  the  POD  and  FAR  are  easy  to  understand  statistics,  but 
they  do  not  account  for  all  available  information  (Flueck,  1987).  The  POD  would  be 
very  high  if  warnings  were  issued  everyday.  The  FAR  would  be  very  low  if  warnings 
were  not  issued  for  marginal  cases. 

A  statistic  that  includes  X,  Y,  and  Z  is  the  critical  success  index: 

CSI  = - - -  (Schaefer,  1990). 

X  +  Y  +  Z 

The  CSI  combines  information  from  both  the  POD  and  FAR  as  shown  in  the  following 
representation: 

(Grenier  &  Halmstad,  1988). 


CSI  = 


1 


1 


POD  I- FAR 


-1 
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Like  the  POD,  a  perfect  CSI  is  1.00.  There  are  still  several  problems  with  the  CSI, 
however.  It  does  not  account  for  correct  negative  forecasts,  ‘W’  from  the  previous  table 
(Flueck,  1987).  Also,  it  is  biased  toward  areas  with  many  events  (Schaefer,  1990). 

A  problem  with  using  verification  statistics  as  the  only  indicator  of  performance 
is  that  the  WFO’s  often  rely  on  non-meteorological  factors  to  improve  their  scores  (Hales 
and  Kelly,  1985).  The  forecaster  is  tempted  to  focus  on  the  probability  of  obtaining 
verification  when  deciding  whether  to  issue  a  warnings  to  a  given  county  (Hales  and 
Kelly,  1985).  For  example,  national  data  from  1979  to  1983  showed  significantly  more 
warnings  were  being  issued  for  highly  populated  counties  (Hales  and  Kelly,  1985). 

A  second  approach  toward  improvement  is  implementing  a  more  aggressive 
search  for  ground  truth  reports  to  verify  warnings  (Hales  and  Kelly,  1985).  In  1983,  the 
Oklahoma  City  (OKC)  WFO  began  an  extensive  post  storm  survey  report  and  Table  1.2 
shows  the  effects  it  had  on  their  verification  statistics  (Hales,  1988). 


Table  1.2.  Verification  Statistics  for  the  OKC  WFO  illustrating  the  effects  of  increased 
efforts  at  gaining  ground  truth  reports  in  1983  (Hales,  1988). 


Year  Events  Counties  POD  FAR  CSI 

Reported  Warned 

1982  258  563  0.508  0.801  0.167 

1983  499  551  0.729  0.508  0.416 

1984  509  568  0.786  0.423  0.499 

1985  558  543  0.774  0.346  0.549 

1986  902  763  0.822  0.287  0.617 


The  number  of  events  reported  increased  four  fold  and  the  FAR  decreased  from  .801  to 
.287  in  the  four  year  test  period.  The  problem  with  this  method  of  improving  verification 


6 


statistics  is  that  there  is  not  enough  time  or  personnel  to  compile  a  flawless  list  of  severe 
weather  events.  Hales  and  Kelly  (1985)  conclude  by  pointing  out  that  the  emphasis 
perhaps  should  be  placed  on  the  rationale  for  issuing  warnings  rather  than  obtaining 
verification. 

In  the  public’s  perception  a  warning  quite  likely  is  justified  if  they 
experience  intense  lightning,  heavy  rainfall  or  even  wind  driven  small 
hail. . .  There  is  certainly  a  question  at  the  present  time  whether  the 
public’s  and  the  forecaster’s  best  interests  are  being  served  as  the 
[verification]  program  is  now  structured  (Hale  and  Kelly,  1985). 

With  some  of  these  limitations  in  mind,  this  thesis  will  analyze  verification  statistics  for 

the  Raleigh  (RDU)  WFO  as  well  as  the  rationale  and  process  of  issuing  warnings. 

1.3  Preliminary  Analysis  of  Doppler  Radar  Capabilities 

An  external  method  of  improving  verification  statistics  is  through  technological 
advances.  The  Doppler  radar  was  developed  in  hopes  of  providing  forecasters  with 
improved  warning  capabilities.  The  first  field  experiment  to  test  the  operational  use  of 
the  Doppler  radar,  the  Joint  Doppler  Operational  Project  (JDOP),  was  conducted  from 
1976-78.  This  project  was  a  combined  effort  of  the  National  Severe  Storms  Laboratory 
(NSSL),  the  National  Weather  Service  (NWS),  the  Air  Weather  Service,  the  Air  Force 
Geophysical  Laboratory  and  the  Federal  Aviation  Administration  (Burgess  and  Devore 
1979). 

According  to  the  Final  Report  on  JDOP:  1976-1978,  operational  tests  showed  the 
“marked  improvement  Doppler  radar  offers  for  early  accurate  identification  of 
thunderstorm  hazards,  especially  tornadoes  and  squall  lines”.  The  report  goes  on  to 
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conclude  that  key  advantages  of  the  Doppler  radar  are:  a  reduced  FAR,  an  enhanced 
POD  and  a  decreased  dependence  on  the  “often  erroneous”  ground  truth  reports  (Staff  of 
JDOP,  1979). 

These  conclusions  were  based  on  a  comparison  between  the  verification  statistics 
for  warnings  issued  by  the  OKC  WFO  and  advisories  issued  by  the  JDOP  forecasters. 
The  advisories  were  based  solely  on  the  Doppler  radar  with  no  public  reports  available  to 
the  forecaster.  The  warnings  issued  by  the  OKC  WFO  were  based  on  ground  truth 
reports,  the  WSR-57,  and  the  Doppler  advisories  issued  by  the  JDOP  staff  (Burgess  & 
Devore  1979).  Verification  information  was  collected  through  records  of  telephone  calls 
and  newspaper  clippings  as  is  normally  done.  However,  there  was  also  an  extremely  in- 
depth  quest  for  ground  truth  using  numerous  follow-up  calls,  the  NSSL  mesonetwork  and 
the  hail  reporting  network  (Staff  of  JDOP,  1979).  According  to  the  Final  Report  on 
JDOP,  “Participants  believe  that  the  1977  and  1978  data  make  up  probably  the  most 
complete  verification  list  ever  documented  for  severe  storms”  (Staff  of  JDOP,  1979). 

This  enhanced  reporting  network  increased  the  number  of  documented  severe 
thunderstorms  and  tornadoes  which  would  generally  boost  the  verification  statistics. 
Table  1.3  shows  the  breakdown  of  the  number  of  warnings  and  advisories  issued  during 
the  JDOP  and  the  corresponding  verification  statistics.  The  Doppler  advisories 
consistently  improved  the  FAR  and  CSI.  The  last  category,  1978  tornadoes,  is  the  only 
circumstance  where  the  POD  dropped  for  the  Doppler  advisories.  This  can  be  explained 
by  the  extremely  high  FAR  and  greater  number  of  warnings  issued  by  the  OKC  NWSO. 
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Table  1.3.  JDOP  Verification  Statistics  (Staff  of  JDOP,  1979) 


1977  SEVERE  STORMS  &  TORNADOES 

OKC  NWSO 

Doppler  Advisories 

Number 

115 

23 

POD 

0.58 

0.75 

FAR 

0.54 

0.22 

CSI 

0.34 

0.62 

1978  SEVERE  STORMS 

OKC  NWSO 

Doppler  Advisories 

Number 

70 

56 

POD 

0.47 

0.7 

FAR 

0.4 

0.16 

CSI 

0.36 

0.62 

1978  TORNADOES 

OKC  NWSO 

Doppler  Advisories 

Number 

42 

7 

POD 

0.75 

0.56 

FAR 

0.79 

0.38 

CSI 

0.2 

0.42 

Lastly,  JDOP  considered  the  justification  for  warnings.  The  forecasters  at  the 
OKC  WFO  were  asked  to  provide  reasons  for  issuing  warnings.  The  results  were 
tabulated  giving  equal  weight  to  each  response  listed.  Table  1.4  is  taken  from  the  Final 
report  on  JDOP  and  shows  the  relative  frequencies  of  each  justification.  Note  that  all 
radar  justifications  are  based  on  the  WSR-57  except  for  the  category  “Doppler  Radar” 
which  came  directly  form  a  JDOP  advisory.  The  OKC  WFO  did  not  have  direct  access  to 
Doppler  data  (Staff  of  JDOP,  1979). 
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Table  1.4.  Justification  for  issuing  warnings  during  JDOP  (Staff  of  JDOP,  1979) 


Severe  Thunderstorm  Warnings 


Reason 

Issued 

Verified 

1. 

Radar  Reflectivities 

43 

20 

2. 

Radar  Tops 

24 

14 

3. 

Reflectivity  &  Tops 

38 

19 

4. 

Public  Reports 

19 

11 

5. 

Doppler  Radar 

16 

10 

Tornado  Warnings 

Reason 

issued 

Verified 

1. 

Public  Reports 

37 

17 

2. 

Radar  (hook  echo) 

27 

5 

3. 

Doppler  Radar 

23 

10 

The  success  of  JDOP  prompted  a  decade  of  development  and  production 
considerations  resulting  in  the  Weather  Surveillance  Radar-1988  Doppler  (WSR-88D) 
(Alberty  and  Crum,  1991).  The  first  operational  WSR-88D  was  in  Norman,  OK  in  March 
of  1991  (Polger  et.  al.  1994).  In  1991  and  1992,  an  operational  analysis  was  done  using 
six  stations  equipped  with  the  WSR-88D  resulting  in  “dramatic,  consistent,  and 
unprecedented  improvements  in  the  accuracies  of  warning  service  and  lead  time” 
(Friday,  1994). 


1.4  The  Modern  National  Weather  Service  Office 

Installing  the  WSR-88D  at  136  WFO’s  is  a  key  component  of  the  NWS’s  program 
called  the  modernization  and  associated  restructuring  (MAR)  aimed  at  improving  severe 
weather  warning  accuracy  and  timeliness  (Polger  et.  al,  1994  and  Klazura  and  Imy, 
1993).  Unfortunately,  the  MAR  has  yielded  less  significant  enhancement  of  mesoscale 
forecast  performance  than  anticipated  (Friday,  1994). 


10 


It  may  just  take  some  time  for  dramatic  improvements  in  forecast  capability.  The 
WSR-88D  is  a  complex  radar  and  requires  both  training  and  experience  to  master 
(Friday,  1994).  The  forecaster  must  be  able  to  “manage  effectively  the  ‘avalanche’  of 
data”  because  of  the  “myriad  number  of  options”  available  on  the  WSR-88D  (Lemon  et. 
al,  1992).  Figure  1.1  illustrates  the  complexity  of  the  warning  process  using  the  WSR- 
88D. 


Choose  20  of  the 
hundreds  of  available 
products 


Look  at  a  few 
from  this  list 


\ 


Evaluate  and 
correctly  interpret 


Augment  this  info  with 
additional  products 
(one  time  requests) 


Make  a 
Decision! 


Figure  1.1.  The  warning  process  using  the  WSR-88D  (Lemon  et.  al,  1992). 


1.5  Raleigh  WFO  and  NCSU  Collaboration 

Another  part  of  the  MAR  of  the  NWS  is  an  emphasis  on  bringing  the  operational 
and  research  communities  together.  The  NWS  had  established  eleven  collaborative 
relationships  between  forecast  offices  and  universities  (Auciello  &  Lavoie,  1993). 
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The  broad  goal  of  collaborative  research  activities  between  National 
Weather  Service  (NWS)  operational  offices  and  universities  is  to  advance 
the  understanding  of  meteorologieal  and  hydrological  phenomena  and  to 
use  this  understanding  to  enhance  weather  warnings,  forecasts,  and  other 
weather  services  to  the  nation  (Auciello  &  Lavoie,  1993). 

It  should  first  be  pointed  out  that  the  atmospheric  science  program  from  North 
Carolina  State  University  (NCSU)  and  the  RDU  WFO  have  been  involved  in 
collaborative  research  for  several  years.  There  have  been  1 1  collaborative  publications 
since  1992  on  topics  ranging  from  winter  precipitation  types  to  severe  weather  outbreaks. 
In  addition,  a  number  of  forecasting  techniques  have  been  developed  through  the 
collaboration  including;  predicting  water  levels  in  the  sounds  of  North  Carolina,  using 
sea  surface  temperatures  to  forecast  the  deepening  rates  of  coastal  storms,  and  predicting 
violent  tornado  outbreaks  in  the  southeastern  United  States  using  a  severe  weather 
paradigm. 

On  a  more  operational  level,  NCSU  faculty  and  students  have  formed  the  Severe 
Weather  Action  Team  (SWAT)  which  has  worked  under  the  direction  of  the  WFO  during 
fifty  severe  weather  events  over  the  past  three  years.  The  purpose  of  the  SWAT  has  been 
both  to  document  and  facilitate  in  the  severe  weather  warning  process.  As  a  result,  both 
researchers  and  forecasters  are  obtaining  a  model  of  how  the  warning  process  unfolds  in 
real  time.  While  the  benefits  of  this  program  cannot  be  objectively  measured,  it  has  both 
provided  support  for  the  forecasters  and  given  researchers  insight  into  the  warning 
process.  Also,  the  experiences  of  the  SWAT  have  charmeled  researchers  toward  practical 
forecast  problems  and  initiated  data  collection  for  future  research. 
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In  order  to  understand  how  the  SWAT  operates,  the  chain  of  command  for  the 
RDU  WFO  must  first  be  presented.  Two  of  the  key  staff  positions  during  severe  weather 
operations  are  the  storm  coordinator  and  the  radar  operator.  The  storm  coordinator  has 
the  ultimate  warning  responsibility.  However,  the  radar  operator  generally  recommends 
that  a  warning  should  be  issued  and  the  storm  coordinator  normally  concurs.  During 
collaborative  events,  members  of  the  SWAT  work  under  the  direction  of  the  storm 
coordinator.  Three  of  the  tasks  performed  by  the  SWAT  members  are:  performing 
mesoscale  analyses,  initiating  phone  calls  requesting  ground  truth  and  documenting  the 
severe  weather  warning  process  from  the  perspective  of  the  radar  operator.  Prior  to  the 
WSR-88D,  the  primary  role  of  the  SWAT  was  operational  support.  However,  currently 
the  team  provides  both  assistance  to  the  forecaster  as  well  as  documentation  of  the 
warning  process.  It  is  this  documentation  done  by  the  SWAT  since  the  RDU  WFO  began 
operational  use  of  the  WSR-88D  in  March  of  1994  that  has  made  this  research  possible. 

1.6  Research  Objectives 

Thousands  of  severe  weather  warnings  are  issued  in  the  United  States  each  year, 
over  a  himdred  of  which  originate  from  the  RDU  WFO.  Each  of  these  warnings  is 
surrounded  by  a  unique  set  of  circumstances  influenced  by  the  meteorological 
characteristics,  the  accuracy  and  timeliness  of  ground  truth  reports,  and  the  forecasters  on 
duty.  Each  forecaster  often  only  works  during  a  few  severe  weather  episodes  each  year 
and  is  unaware  of  how  the  warning  process  unfolds  when  he  or  she  was  not  working. 
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Therefore,  the  objective  of  this  research  is  to  analyze  how  warnings  are  issued  at  the 
RDUWFO. 

In  order  to  attain  this  objective,  the  research  has  been  divided  into  three  phases. 

1)  Chapter  two  covers  the  data  organization  phase.  First,  details  on  the  data  sets  utilized 
in  this  research  will  be  provided.  Then  two  case  studies  of  severe  weather  episodes  will 
be  given  to  illustrate  how  the  warning  process  unfolds  in  real  time.  These  case  studies 
will  also  serve  as  examples  of  the  documentation  done  by  the  SWAT. 

2)  Then  chapter  three  will  then  move  to  a  more  general  view  of  severe  weather 
operations  through  a  schematic  representation  of  the  warning  process. 

3)  Finally,  a  thorough  discussion  of  the  verification  statistics  is  given  in  chapter  four. 
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2.  DATA  ORGANIZATION 


The  data  for  this  study  come  from  four  sources:  the  Warning  Logs,  Action  Logs, 
SWAT  reports,  and  Storm  Data.  The  Warning  Log  is  the  WFO’s  listing  of  all  warnings 
issued  during  an  episode  including  the  counties  warned,  the  type  of  warning  issued,  and 
the  time  it  was  issued  and  expired.  For  this  research,  only  severe  thunderstorm  (SVR) 
and  tornado  (TOR)  warnings  will  be  considered.  The  Action  Log  is  the  NWS’s  hand 
written  record  of  the  initiated  and  received  phone  calls  in  conjunction  with  the  episode. 
The  SWAT  reports  (SWATR)  are  the  documented  timelines  of  the  activity  during  severe 
weather  episodes  from  the  perspective  of  the  WSR-88D  operator.  Finally,  Storm  Data  is 
used  to  verify  warnings  as  discussed  in  the  first  chapter.  The  spreadsheet  organization 
used  in  this  research  is  provided  in  Appendix  7.2  and  some  excerpts  from  the 
spreadsheets  are  given  in  Appendix  7.3.  This  chapter  first  describes  the  data  set  for  each 
phase  of  the  analysis  and  then  provides  two  case  studies  as  real  time  illustrations  of  the 
warning  process  and  examples  of  the  SWATR. 

2.1  The  Data  Set  for  Each  Phase  of  the  Analysis 

There  are  two  data  sets  in  this  research  based  on  the  information  available:  the 
Process  Data  Set  (PDS)  and  Statistical  Data  Set  (SDS). 

The  SWATR  are  the  primary  data  source  for  the  analysis  of  the  warning  process 
which  includes  information  on  the  rationale  behind  issuing  warnings.  The  SWATR 
contain  detailed  documentation  of  the  sequence  of  events  during  the  severe  weather 
episode.  However,  when  the  WSR-88D  was  first  used  at  the  RDU  WFO,  the  SWAT 
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recorders  were  not  familiar  with  the  products  available  on  the  new  radar,  and  therefore 
the  documentation  was  less  complete.  In  light  of  this  problem,  the  data  set  for  the 
process  section  of  this  research  has  been  limited  to  the  1995  SWATR,  the  seeond  severe 
weather  season  with  the  WSR-88D. 

Additional  inconsistencies  in  the  SWATR  caused  the  process  data  set  to  be 
reduced  further.  Each  SWAT  recorder  focused  on  different  elements  of  the  warning 
process.  Also,  some  radar  operators  verbalized  their  thought  process  simplifying  the 
documentation  procedure,  while  it  was  more  difficult  to  capture  the  decision  making 
process  for  other  operators.  Finally,  the  SWAT  recorder  had  a  difficult  time  keeping  up 
with  the  documentation  during  very  active  episodes.  Therefore,  the  warnings  logged  in 
the  1995  SWATR  had  to  be  screened  and  only  the  thoroughly  documented  warnings  have 
been  included  in  the  PDS.  Also  because  the  PDS  will  be  used  for  the  analysis  of  the 
warning  process,  multiple  warnings  issued  to  neighboring  counties  are  treated  as 
individual  warning  entries.  The  resulting  PDS  includes  a  total  of  18  episodes 
encompassing  48  severe  thunderstorm  and  20  tornado  warnings.  This  is  over  90  percent 
of  the  1995  warnings  with  SWATR  and  33  percent  of  the  total  warnings  issued  by  the 
RDU  WFO  in  1995.  Figure  2.1  shows  the  severe  weather  episodes  included  in  the  PDS. 
Each  asterisks  represents  a  warning  in  the  PDS  while  the  lines  indicate  additional 
warnings  that  were  not  included  in  the  PDS.  Notice  that  the  PDS  includes  several 
episodes  with  numerous  warnings  as  well  as  six  cases  where  less  than  four  warnings  were 
issued. 
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The  Events  in  the  Process  Data  Set  (PDS) 

Time  (local— EST  or  EDT  as  applicable) 


Dates 

06  12  18  24 

1  1  1  1 

3/08/95 

1  1  1  1 

if  *  :<c 

5/01/95 

*  ♦♦ 

5/10/95 

*  *  II 

5/13/95 

♦  ♦  +  *  * 

5/15/95 

1  * 

5/17/95 

* 

5/19/95  AM 

II  mil  III  **  **4c*3fc4c* 

5/19/95  PM 

if  if. 

5/26/95 

♦ 

6/09/95 

6/11/95 

6/12/95 

ifif-ifi^iifififif^  1 

6/16/95 

**  if  ** 

7/21/95 

1  11  1  **  1  1  II 

10/05/95  AM 

1  *  * 

10/05/95  PM 

if  if  if 

10/27/95 

|||||*«***  if  if  1 

11/11/95 

III  II**  ** 

Figure  2.1.  Pictorial  representation  of  the  PDS.  The  warnings  in  the  PDS  are  indicated 
by  the  asterisks  while  the  lines  represent  additional  warnings  during  the  severe  weather 
episode. 


For  the  analysis  that  does  not  require  SWATR,  the  Statistical  Data  Set  (SDS)  was 
introduced.  This  data  set  includes  information  from  the  Warning  Log,  Action  Log,  and 
Storm  Data  and  was  useful  in  analyzing  the  verification  statistics  and  importance  of 
ground  truth  reports.  The  SDS  is  composed  of  the  PDS  as  well  as  any  additional 
warnings  issued  by  the  RDU  WFO  during  the  severe  weather  season  after  the  WSR-88D 
was  operational.  This  includes  episodes  from  June  through  August  of  1994  and  from 
May  through  August  of  1995.  A  total  of  327  severe  thunderstorm  warnings  and  24 
tornado  warnings  are  in  the  SDS. 
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2.2  Case  Studies 


In  order  to  illustrate  the  type  of  information  available  in  the  PDS  as  well  as  give 
the  reader  an  idea  of  how  the  warning  process  progresses  in  real  time,  two  case  studies 
have  been  included.  The  bracketed  information  in  the  cases  relates  to  the  warning 
process  which  will  be  presented  in  chapter  three.  The  reader  is  encouraged  to  read 
through  the  cases  first  to  get  an  overall  idea  of  the  activity  at  the  WSR-88D  Principal 
User  Processor  (PUP)  during  a  severe  weather  episode.  Then  he  or  she  should  refer  back 
to  the  cases  while  reading  chapter  three  to  see  how  these  specific  examples  fit  into  the 
general  model  of  the  warning  process. 

The  detailed  summaries  of  the  environment  leading  to  the  episodes  come  directly 
from  the  SWATR.  The  radar  images  have  been  included  to  show  a  few  of  the  WSR-88D 
products  utilized  and  to  give  the  reader  a  better  concept  of  the  warning  process  from  the 
perspective  of  the  WSR-88D  operator. 

The  first  case  is  taken  from  October  5,  1995  and  will  illustrate  the  progression  of 
events  during  a  slow  paced  case  where  a  cell  develops,  crosses  through  two  counties  and 
then  dissipates.  At  0900  UTC,  a  severe  thunderstorm  watch  was  issued  for  much  of 
southern  North  Carolina.  On  the  synoptic  scale,  there  is  warm  moist  air  over  North 
Carolina  associated  with  the  remnants  of  Hurricane  Opal  as  shown  on  the  surface 
analysis  given  in  Figure  2.2.  Notice  the  cyclone  center  is  over  eastern  Tennessee  and  is 
tracking  northward.  Also,  the  surface  dew  points  are  in  the  mid  70’s°  F  across  much  of 
the  region.  A  warm  front  has  moved  northwest  during  the  night  placing  the  RWA  in 
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tropical  air.  The  1200  UTC  (7:00  AM  EST)  Greensboro  sounding  given  in  Figure  2.3  is 
nearly  moist  adiabatic  but  there  is  strong  veering  of  the  winds  with  height  in  the  lowest 
levels  of  the  atmosphere.  Thus,  the  helicity  is  high  in  the  lowest  layers.  Based  on  a 
surface  parcel  of  T  =  25°C  and  Tj  =  24“C,  the  lifted  index  is  -3  and  the  CAPE  is  933  J/kg. 


Figure  2.2.  Surface  Analysis  for  1200  UTC  05  Oct.  1995  (Daily  Weather  Maps). 
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Figure  2.3.  1200  UTC  05  Oct.  1995  skew  T-log  p  plot  of  Greensboro,  NC  (GSO).  The 
dashed  line  is  the  dew  point,  the  thin  solid  line  is  the  temperature  and  the  bold  line 
represents  the  parcels  path. 


At  about  07:00  AM  EST,  thunderstorms  with  tomadic  potential  are  moving  into 
the  RWA  from  the  south.  Figure  2.4  shows  the  locations  mentioned  in  the  text.  The 
times  are  given  in  EST.  Tornado  warnings  are  issued  for  Anson  and  Cumberland 
Counties  prior  to  the  SWATR  segment  that  will  now  be  presented. 
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Figure  2.4.  Location  map  for  the  05  Oct.  1995  case. 

08:29  The  radar  operator  is  looking  at  the  relative  velocity  over  central  Chatham 
County.  The  gate-to-gate  shear  is  26  knots  at  7,700  ft.  [Initiator] 

08:36  The  shear  is  now  25  knots  for  the  above  cell.  The  four  panel  relative  velocity  and 
reflectivity  products  for  1230  UTC  have  been  included  as  Figs.  2.5  and  2.6.  The 
radar  operators  attention  is  focused  on  cell  A  in  central  Chatham  County  which  is 
labeled  at  the  1.5  degree  elevation  angle  in  figures  2.5  and  2.6.  Notice  that  the 
relative  velocity  for  cell  A  indicates  35  knots  of  gate-to-gate  shear  at  mid  levels. 
The  shear  in  Chatham  County  is  well  defined  at  both  the  0.5  and  1.5  degree 
elevation  angles.  [Trigger]  The  radar  is  located  to  the  southeast  of  each  velocity 
panel  as  shown  in  Appendix  7.1.  Figure  2.6  shows  a  reflectivity  maximum 
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Figure  2.5.  WSR-88D  four-panel  velocity  for  Chatham  County  at  1230  UTC  (08:30  EST)  on  05  Oct.  1995. 
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Figure  2.6.  WSR-88D  four-panel  reflectivity  for  Chatham  County  at  1230  UTC  (08:30  EST)  on  05  Oct.  1995. 


of  50  to  55  dBZ  in  cell  A.  Based  on  the  increased  shear  values,  the  decision  is 
made  to  issue  a  warning. 

08:37  The  tornado  warning  is  issued  for  Chatham  County. 

08:40  The  radar  operator  is  looking  at  the  base  reflectivity.  There  is  a  weak  pendant  on 
the  south  side  of  the  cell  in  north-central  Chatham  County.  A  call  is  initiated  to 
Chatham  County.  They  report  trees  down.  [Initiator] 

08:42  A  report  is  received  of  a  possible  tornado  near  Pittsboro  in  Chatham  County. 

08:45  The  Blacksburg,  VA  NWS  Office  calls  regarding  the  conditions  in  the  RWA. 

08:47  The  relative  velocity  magnified  four  times  shows  19  knots  of  shear  in  the 
Chatham  County  cell  which  is  now  entering  Orange  County. 

08:48  The  radar  operator  is  continuing  to  monitor  the  relative  velocity  at  four-times- 
magnification.  The  gate-to-gate  shear  is  now  up  to  25  knots  in  the  above  cell. 
[Trigger] 

08:50  The  radar  operator  recommends  that  a  warning  be  issued  for  Orange  County 
based  on  the  shear  values  and  the  reports  verifying  the  Chatham  County  warning. 

08:51  The  Orange  Coimty  tornado  warning  is  issued. 

08:52  The  cell  has  a  reflectivity  structure  suggestive  of  a  supercell. 

08:53  The  cell  is  now  positioned  in  southwest  Orange  County  and  still  shows  good 
organization  on  the  four  panel  reflectivity. 

08:54  The  relative  velocity  shows  19  knots  of  shear.  A  call  is  initiated  to  Orange 
County. 

08:55  There  is  now  21  knots  of  gate-to-gate  shear  in  the  above  cell. 
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08:58  It  is  noted  that  cells  are  moving  very  rapidly.  Chatham  County  is  now  clear  and 
the  warning  is  being  pulled  off  of  the  weather  radio  tape.  The  relative  velocity 
for  1254  UTC  scan  reveals  19  knots  of  shear. 

09;  1 1  The  radar  operator  is  looking  at  the  relative  velocity  for  the  entire  warning  area 
and  notes  that  the  only  organized  shear  is  in  Orange  County. 

09: 15  The  shear  in  Orange  County  extends  to  10,000  feet. 

09:20  The  relative  velocity  on  the  13:17  UTC  scan  shows  that  the  cell  in  Orange  Coxmty 
seems  to  becoming  less  organized. 


The  second  case  is  taken  from  June  16, 1995  and  illustrates  a  much  more  active 
severe  weather  episode  with  four  warnings  issued  for  three  different  cells  in  less  than 
thirty  minutes.  For  the  previous  five  days,  severe  weather  had  developed  in  unstable  air 
ahead  of  an  approaching  cold  front.  So  far,  however,  the  convection  has  produced  only 
marginal  severe  weather  with  mostly  scattered  wind  damage  and  some  dime-size  hail. 
The  surface  analysis  given  in  Fig.  2.7  shows  the  position  of  a  surface  cold  front  at  2000 
UTC  (16:00  EDT)  for  this  case.  Ahead  of  the  front  there  has  been  some  clearing  and 
destabilization.  Also  notice  the  abundant  moisture  with  dew  points  in  the  70 ’s  over 
much  of  the  eastern  Carolinas-in  the  eastern  part  of  the  RWA. 
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Figure  2.7.  Surface  analysis  for  2000  UTC  12  Jun.  1995 

Figure  2.8  shows  the  1200  UTC  sounding  for  Moorehead  City,  NC.  This 
sounding  was  selected  because  it  was  ahead  of  the  rain  cooled  air  and  was  the  last 
sounding  available  to  the  operational  forecaster.  Even  without  surface  modification,  the 
CAPE  is  2478  m'/s*  which  is  supportive  of  strong  convection.  However,  the  storm 
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relative  helicity  is  only  24  mVs^  a  value  not  supportive  of  supercells.  Thus,  pulse-type 
convection  with  possible  hail  and  straight  line  wind  damage  are  anticipated. 


Figure  2.8.  1200  UTC  12  Jun.  1995  skew  T-log p  plot  of  Moorehead  City,  NC  (MHX). 

Now  the  SWATR  for  the  first  four  warnings  vdll  be  presented  with  times  given  in 
EDT.  Figure  2.9  is  the  location  map  for  this  example.  In  order  to  keep  track  of  all  the 
severe  and  potentially  severe  cells,  several  convective  lines  and  cells  are  labeled  in  Fig. 
2.11.  The  labeling  convention  \vitl  be  used  throughout  this  excerpt  from  the  SWATR. 
However,  the  reader  is  cautioned  to  keep  in  mind  that  the  lines  and  cells  are  evolving 
during  the  episode  and  the  changing  locations  of  the  cells  will  be  noted  in  the  excerpt. 
Finally,  Fig.  2. 10  is  a  flow  chart  of  the  warnings  issued  during  the  excerpt. 
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Figure  2.9.  Location  map  for  12  Jun.  1995  case. 


Figure  2.10.  Flow  chart  of  the  warnings  issued  during  the  12  Jun.  1995  case.  All 
warnings  are  severe  thunderstorm  warnings.  The  times  given  are  in  EDT. 
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16:44  The  RDU  WFO  received  word  that  the  Charlotte  WFO  had  issued  a  severe 
thunderstorm  warning  for  cell  A  in  Union  County  immediately  southwest  of 
Stanly  County. 

16:45  Cell  A  is  located  six  or  seven  miles  south  of  Albamarle  in  Stanly  County  and  has 
a  reflectivity  of  65  dBZ.  [Trigger]  The  decision  is  made  to  issue  a  severe 
thunderstorm  warning  for  Stanly  County.  Note  that  Figure  2. 10  is  the  composite 
reflectivity  for  the  2047  UTC  (16:47  EDT)  scan  showing  that  cell  A  is  a  small 
intense  cell  entering  western  Stanly  County. 

16:48  The  warning  for  cell  A  is  issued  for  Stanly  County.  The  radar  operator  is  looking 
at  the  storm  relative  velocity  in  line  C  over  Hoke  County. 

16:49  Cell  B  in  northwest  Vance  Coimty  has  reflectivity  values  from  55  to  60  dBZ.  The 
radar  operator  obtains  a  cross  section  of  the  cell  showing  a  maximum  core  of  65 
dBZ  with  good  southward  overhang.  [Trigger]  The  decision  is  made  to  issue  a 
warning  for  northern  Vance  County. 

16:50  A  mesocyclone  alert  sounds  for  a  strong  cell  in  Cumberland  County  in  the 
northeastern  section  of  line  C. 

16:52  The  severe  thunderstorm  warning  on  cell  B  for  Vance  County  is  transmitted. 

16:53  The  radar  operator  is  looking  at  the  four-panel  velocity  in  Cumberland  County  in 
the  northeastern  region  of  line  C. 

16:54  Ground  truth  reports  are  solicited  from  Cumberland  and  Harnett  Counties  in  the 
northeastern  regions  of  line  C.  There  have  been  no  hail  or  damage  reports.  A 
severe  thunderstorm  warning  is  prepared  for  Granville  County  as  the  central 
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Figure  2. 11.  WSR-88D  composite  reflectivity  for  2047  UTC  (16:47  EDT)  on  12  Jun.  1995. 


section  of  line  D  approaches  from  the  west.  The  warning  will  be  held  until  a 
decision  is  made  whether  or  not  it  should  be  issued. 

16:55  A  storm  cell  is  noted  over  east-central  Wake  County. 

16:56  Granville  County  reports  no  severe  weather  associated  with  line  D. 

16:57  The  VIL  has  decreased  on  the  Granville  County  cell.  Because  of  this  decrease  in 
the  VIL  and  the  above  ground  truth  report,  the  decision  is  made  not  to  warn 
Granville  County  for  line  D. 

16:58  A  special  weather  statement  is  started  for  Wake,  Johnston  and  Harnett  Counties. 

17:01  Looking  at  the  northeastern  regions  of  line  C,  the  four  panel  reflectivity  for  Hoke 
County  is  not  impressive. 

17:02  A  hail  alert  is  present  for  Granville  County  on  line  D.  Granville  County  is  called 
for  reports. 

17:05  The  reflectivity  values  are  now  65  dBZ  in  a  cell  in  western  Hoke  County  in  the 
northeastern  region  of  line  C  [Trigger].  The  decision  is  made  to  warn  Hoke 
County. 

17:06  A  mesocyclone  alert  sounds  for  northern  Cumberland  County  in  the  northeastern 
region  of  line  C. 

17:07  The  VIL  is  now  45  to  50  kg/m^  in  Hoke  Coimty  and  northern  Cumberland  County 
in  the  northeastern  region  of  line  C. 

17:08  The  Hoke  County  warning  is  transmitted  for  a  segment  of  line  C. 
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17:09  Vance  County  is  called  for  damage  reports  associated  with  cell  B.  There  is  3-D 
correlated  shear  indicated  east  of  Williamsboro  in  Vance  County. 

17: 1 1  Stanly  County  is  called  for  damage  reports  form  cell  A. 

17:12  The  radar  operator  is  looking  at  a  cross-section  of  the  cell  in  northern 
Cumberland  County  on  line  C.  It  does  not  look  impressive. 

17:13  The  composite  reflectivity  shows  60  dBZ  or  more  on  line  C  in  northern 
Cumberland  Coimty.  [Initiator] 

17:14  The  radar  operator  is  analyzing  the  four  panel  reflectivity  for  northern 
Cumberland  County.  There  is  a  report  of  3/4  inch  hail  from  Hoke  County  which 
is  also  on  line  C. 

17: 16  The  VIL  is  50  kg/m^  in  southwest  Hoke  County  on  the  coimty  line. 

17:19  The  radar  operator  is  considering  warning  Harnett  County  for  cell  E  shown  on 
Fig.  2.11.  A  call  is  initiated  to  Harnett  County  for  possible  reports.  A  cross 
section  of  cell  E  in  Harnett  County  is  given  in  Fig.  2. 12  and  shows  50  dBZ 
extending  to  about  20,000  feet.  The  core  is  around  55  dBZ  and  shows  some 
overhang.  [Trigger] 

1 7:20  The  decision  is  made  to  issue  a  severe  thunderstorm  warning  for  Harnett  County. 

17:21  There  is  a  report  of  marble  size  hail  in  Scotland  County  associated  with  line  C. 

17:22  A  warning  is  prepared  for  Scotland  County  and  is  held  until  the  decision  is  made 
whether  or  not  a  warning  should  be  issued. 

17:23  The  Harnett  County  warning  is  transmitted. 
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Figure  2. 12.  WSR-88D  composite  reflectivity  for  2117  UTC  (17:17  EDT)  on  12  Jun.  1995. 
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Figure  2. 13.  A  southwest  to  northeast  WSR-88D  cross  section  of  cell  E  indicated  in  Fig.  2. 12. 


The  warning  was  never  issued  for  Scotland  County  because  conditions  did  not 
intensify.  However,  an  additional  seven  warnings  were  issued  during  this  severe  weather 
episode. 

These  two  case  studies  have  been  included  to  illustrate  the  activity  at  the  WSR- 
88D  PUP  during  a  severe  weather  episode.  The  first  case  was  a  slow  paced  event  where 
the  radar  operator’s  attention  was  entirely  focused  on  an  individual  intense  cell.  In 
contrast,  the  second  case  was  included  to  illustrate  the  opposite  extreme  where  the  radar 
operator  must  quickly  investigate  multiple  severe  cells.  The  activity  level  for  most 
severe  weather  episodes  fall  somewhere  between  the  two  cases  provided  in  this  chapter. 
The  remainder  of  this  report  will  be  a  more  general  examination  of  the  warning  process 
using  these  cases  to  illustrate  the  points  made. 


3.  SCHEMATIC  REPRESENTATION  OF  THE  WARNING  PROCESS 


As  illustrated  by  the  case  studies  in  the  previous  chapter,  the  warning  process 
includes  a  complex  series  of  events  leading  a  forecaster  to  a  decision  to  (or  not  to)  issue  a 
warning.  Keep  in  mind  throughout  this  chapter  that  the  forecaster  is  often  dealing  with 
multiple  warnings  at  different  phases  in  the  warning  process  as  shown  by  the  second 
case.  However,  to  gain  an  understanding  of  how  the  process  operates,  the  labyrinth  has 
been  simplified  to  include  the  sequence  of  key  events  common  to  in  the  issuing  of  all 
warnings. 

The  schematic  representation  of  the  warning  process  presented  in  Fig.  3.1  was 
developed  from  the  PDS  and  illustrates  how  the  forecaster  progresses  from  monitoring 
the  situation  to  issuing  a  warning.  Usually  some  event,  defined  here  as  the  initiator, 
prompts  the  radar  operator  to  perform  further  investigation  on  a  given  cell  and  to 
consider  issuing  a  warning.  In  some  cases,  the  decision  is  made  not  to  issue  a  warning. 
Otherwise,  a  second  key  event,  called  the  trigger,  leads  to  drives  the  decision  to  issue  the 
warning.  In  some  circumstances,  the  warning  has  only  a  trigger  followed  by  the 
immediate  issuing  of  the  warning  as  shown  by  both  of  the  dashed  paths  in  Fig.  3.1.  This 
class  of  warning  will  be  called  the  immediate  trigger  warnings.  After  the  warning  has 
been  issued,  ground  truth  reports  are  commonly  sought  which  can  lead  to  the  verification 
of  the  warning. 
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Figure  3.1.  This  schematic  representation  of  the  warning  process  traces  the  sequence  of 
events  leading  a  forecaster  to  a  warn  or  no-wam  decision  for  an  individual  cell.  The 
numbers  represent  the  number  of  times  a  forecaster  was  along  each  path.  The  asterisk  in 
the  verify  warning  box  represents  the  six  warnings  whose  initiator  was  a  previous 
verification  for  a  preceding  warning  that  was  not  documented.  The  bold  track  is  taken 
most  often.  The  two  dotted  cycles  are  immediate  trigger  warnings. 

The  utilized  radar  products  and  ground  truth  reports  provide  information  that 
serves  as  the  initiator,  trigger  and  decisions  not  to  warn.  Table  3.1  lists  the  broad  and 
subcategories  of  this  information.  Note  that  the  first  three  subcategories  are  a  breakdown 
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of  the  reflectivity,  ‘R’,  category,  the  next  two  subcategories  are  derived  from  the  velocity, 
‘Ve’,  category,  and  finally  the  last  three  subcategories  are  the  components  of  the  ground 
truth,  ‘G’,  category.  In  analyzing  the  initiator  and  trigger,  the  text  will  refer  to  these 
categories  of  activities  as  the  initiators  and  the  triggers.  Clearly,  specific  information  is 
lost  by  categorizing  activities  and  therefore  Appendix  7.3  contains  the  detailed  listing  of 
initiators,  triggers,  and  decisions  not  to  warn. 


Table  3.1.  The  broad  categories  and  subcategories  used  in  the  analysis  of  the  initiator, 
trigger  and  decisions  not  to  warn. 


_ BROAD  CATEGORIES _ 

R  Reflectivity  based  Doppler  radar  products 
Ve  Velocity  based  Doppler  radar  products 
G  Ground  truth  reports 

_ SUBCATEGORIES _ 

R  Base  or  composite  reflectivity 
Vi  Vertically  Integrated  Liquid  (VIL) 

X  Reflectivity  cross  section 

M  Mesocyclone 

Ve  Any  base  or  Storm  Relative  Velocity  Product 
T  In  the  track  of  a  previously  verified  cell 

C+  Immediate  call  with  a  report  of  a  severe  thunderstorm  or  tornado  in  or  very 
near  the  county  warned 

C-  Immediate  call  with  a  report  of  weather  that  does  not  meet  severe 
thunderstorm  or  tornado  criteria  in  or  very  near  the  warned  county 


Notice  from  Table  3.1  that  one  of  the  categories  of  activity  is  ground  truth  reports.  To 
avoid  confusion,  it  is  important  to  note  that  the  “Call  for  Ground  Truth”  box  in  Figure  3.1 
is  located  in  the  most  common  position  for  reports  to  be  sought,  but  it  is  not  the  only  time 
calls  occur.  Throughout  the  warning  process,  the  NWS  personnel  are  initiating  and 
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receiving  ground  truth  reports  from  the  public  as  shown  in  the  cases  presented  in  chapter 
two.  These  ground  truth  reports  often  become  the  initiator,  trigger  or  rationale  for  a 
decision  not  to  warn. 

The  remainder  of  this  chapter  steps  through  each  block  of  Fig.  3.1  giving  both  a 
qualitative  and  quantitative  analysis  of  the  warning  process.  In  numeric  computations, 
the  data  set  is  often  a  subset  of  the  PDS  because  of  incomplete  documentation.  However, 
in  order  for  the  totals  entering  the  blocks  to  balance  with  those  exiting,  the  numbers  in 
Fig.  3. 1  include  the  entire  PDS. 

3.1  Monitor  the  Situation 

When  there  is  a  threat  of  severe  weather,  the  radar  operator  monitors  the  situation 
throughout  the  county  warning  area.  This  phase  can  last  for  hours  or  quickly  become 
focused  with  the  development  of  an  initiator  or  a  trigger. 

3.2  The  Initiator 

The  initiator  is  the  event  that  prompts  the  radar  operator  to  investigate  a 
potentially  severe  cell  and  consider  issuing  a  warning.  To  ensure  statistical 
independence,  each  initiator  is  linked  to  only  one  of  the  categories  listed  in  Table  3.1.  It 
is  important  to  note  that  the  initiator  is  only  separated  from  the  trigger  in  cases  where 
there  was  at  least  two  minutes  of  investigation  prior  to  the  decision  to  issue  the  warning. 
Often  the  SWATR  clearly  states  the  radar  operator  is  considering  issuing  a  warning,  and 
hence  the  initiator  can  easily  be  determined. 
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However,  for  other  warnings  it  is  more  difficult  to  determine  the  exact  initiator, 
and  therefore  it  must  be  determined  indirectly  by  proceeding  backwards  from  an  issued 
warning  or  documented  decision  not  to  warn.  In  these  cases,  the  initiator  is  the  first 
activity  documented  where  the  radar  operator  is  investigating  an  individual  cell  and  is 
considering  issuing  a  warning.  An  example  of  an  initiator  found  indirectly  is  the  initiator 
for  the  Harnett  County  warning  in  case  two  at  17:13  EDT.  There  is  ambiguity  in  the 
determination  of  this  initiator.  The  radar  operator  is  focusing  on  the  Harnett  County  cell 
when  a  cross  section  was  performed  at  17:12  EDT  .  However,  since  the  cross-section  is 
not  impressive,  it  does  not  prompt  the  radar  operator  to  consider  issuing  a  warning  and 
hence  is  not  the  initiator. 

Finally,  there  are  15  cases  where  the  documentation  indicates  the  radar  operator  is 
investigating  a  cell  and  considering  issuing  a  warning  but  the  SWATR  does  not  specify 
the  initiator.  Therefore,  out  of  the  81  cases  in  Fig.  3.1,  a  total  of  66  warnings,  41  severe 
thunderstorm  and  25  tornado  warnings  are  included  in  the  following  analysis.  Figure  3.2 
shows  the  distribution  of  initiators  for  issuing  severe  thunderstorm  and  tornado  warnings. 
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Figure  3.2.  Initiators  for  severe  thunderstorm  (SVR)  and  tornado  (TOR)  warnings. 
Figure  3.2A  used  the  broad  categories  while  Fig.  3.2B  utilizes  the  subcategories.  Table 
3.1  defines  the  categories.  A  total  of  41  severe  thunderstorm  warnings  and  25  tornado 
warnings  are  included.  The  numbers  above  the  columns  are  the  column  totals  which  will 
be  used  in  the  statistical  analysis  that  follows. 


A  contingency  table  was  used  to  analyze  the  statistical  contrasts  between  the 
initiators  for  both  severe  thunderstorm  and  tornado  warnings.  The  following  example 
illustrates  this  test  for  severe  thunderstorm  warnings  using  the  broad  categories  of 
activities.  The  null  hypothesis,  for  a  statistical  test  is  the  statement  to  be  disproved. 
In  this  case.  Ho  states  that  the  probability  of  severe  thunderstorms  having  each  of  the 
three  initiators  is  the  same.  Before  the  test  can  be  presented,  the  following  variables 
must  be  defined  (Mendenhall  and  Sincich,  1992): 

Pi  =  the  hypothesized  probability  of  being  in  category  i.  Under  the  null  hypothesis. 


Pi  must  be 


for  each  initiator. 
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obs  =  the  observed  number  of  warnings  with  each  initiator.  The  figure  above  shows 

that  reflectivity  is  the  initiator  for  17  warnings,  velocity  for  7  warnings,  and 
ground  truth  for  17  warnings, 
n  =  the  sample  size.  In  this  case,  n=41. 

f  1)  2 

exp  =  n  (Pi)  =  41  1^— J  =  13  — ,  the  expected  count  for  each  initiator. 

Now,  the  following  assumptions  must  be  met  for  this  test  to  be  used  (Mendenhall  and 
Sincich,  1992); 

1)  The  trials  must  be  identical.  In  this  case,  each  warning  used  in  the  initiator  analysis  is 
a  trial.  The  initiators  for  severe  thunderstorm  and  tornado  warnings  have  been  separated 
in  this  analysis  in  an  attempt  to  make  the  trials  identical.  The  PDS  is  too  small  to  make 
further  divisions. 

2)  There  are  k  possible  outcomes  for  each  trial.  For  this  example  there  are  three 
initiators  and  hence  k=3. 

3)  The  hypothesized  probabilities  must  sum  to  one,  +— =1 

'  3  3  3 

4)  The  trials  must  be  independent.  This  is  why  only  one  category  of  activity  is  the 
initiator  for  each  warning. 

5)  The  random  variables  must  be  counts  for  each  cell. 

6)  The  sampling  size,  n,  must  be  “large”.  The  test  of  this  assumption  is  as  follows, 

2 

exp  >  5,  for  all  k  categories.  For  this  example,  exp  =  13—  for  all  initiators  and  thus  this 
final  assumption  is  met. 
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Now  that  the  variables  have  been  defined  and  the  assumptions  met,  the  statistical 
test  can  be  presented.  Table  3.2  is  the  contingency  table  for  this  example. 


Table  3.2.  The  contingency  table  for  this  example-severe  thunderstorm  warnings  with 
the  three  broad  categories  of  initiators.  The  variables  are  defined  in  the  text  above. 

OCCURRENCES 


INITIATORS 


Obs 

Exp=n(Pi) 

R 

17 

13.67 

Ve 

7 

13.67 

G 

17 

13.67 

Total 

41 

The  test  statistic,  X^,  has  approximately  a  Chi  Square  distribution  for  large  sample  sizes 
and  is  defined  as  follows: 

,^,  (ofe-exp)^  (17-13.67)^  ^  (7-13.67)^  ^  (17-13.67)^  ^ 

*  exp  13.67.  13.67  13.67 

vrith  (k-1)  =  2  degrees  of  freedom. 

To  reject  Hq,  X  must  be  greater  than  I  crit  where  I  cht  is  the  critical  value  of  the  Chi 
Square  distribution  for  the  given  confidence  level  and  degrees  of  freedom.  In  this 
statistical  test  and  for  the  remainder  of  this  report,  the  confidence  level  will  be  0.05.  For 
this  case,  I^cnt  =  5.99  and  hence  Ho  caimot  be  rejected.  Therefore,  the  conclusion  is  that 
there  is  not  statistically  significant  evidence  of  any  differences  in  the  initiator 
probabilities  for  severe  thunderstorm  warnings. 

This  same  analysis  can  be  used  with  the  subcategories  and  for  tornado  warnings 
as  summarized  in  Table  3.3. 
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Table  3.3.  The  statistical  results  for  initiators.  The  test  determines  the  differences 
between  the  broad  and  subcategories  of  initiators  for  SVR  and  TOR  warnings. 


Statistical  Results:  The  Initiator 


n 

WM3M 

exp 

- 7^ - 

cnt 

Conclusion 

SVR 

broad  categories 

41 

0.333 

13.67 

4.88 

5.99 

Fail  to  Reject  Ho 

subcategories 

41 

0.125 

5.125 

6.03 

14.1 

Fail  to  Reject  Ho 

TOR 

broad  categories 

20 

0.333 

8.33 

4.16 

5.99 

Fail  to  Reject  Ho 

subcategories 

20 

0.125 

3.125 

Insufficient  Sample  Size 

The  first  test  summarizes  the  example  given  above.  The  next  two  tests  have  the  same 
conclusion  as  the  example,  a  failure  to  reject  Ho.  Notice  that  the  last  test  was  not 
completed  because  of  a  failure  to  meet  the  assumption  of  a  “large”  sample  size.  So  it  is 
found  that  no  statistically  significant  conclusions  can  be  made  concerning  the  initiators. 
Therefore,  none  of  the  initiators  appear  to  be  used  more  than  others. 

3.3  Investigate  the  Cell 

By  definition,  the  only  exit  from  the  initiator  block  is  to  investigate  the  cell.  The 
investigation  time  is  defined  as  the  time  between  the  initiator  and  the  trigger.  The 
average  investigation  time  is  7.2  minutes.  However,  this  average  is  skewed  high  because 
of  a  few  long  periods  of  investigation.  The  median  of  the  distribution  is  five  minutes  and 
the  mode  is  only  three  minutes.  Figure  3.3  illustrates  the  distribution  of  investigation 
times. 
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Investigation  Time 


Time  (min.) 

Figure  3.3.  The  distribution  of  investigation  times. 

Long  investigation  times  are  not  necessarily  undesirable  but  simply  indicate  that 
the  radar  operator  was  focused  on  an  individual  cell  for  a  long  period  of  time.  During 
episodes  with  many  warnings,  the  radar  operator  has  less  time  to  focus  on  individual  cells 
and  therefore  the  investigation  times  are  generally  shorter. 

Out  of  the  “Investigate  the  Cell”  phase,  the  radar  operator  either  encounters  a 
trigger  and  recommends  that  a  warning  be  issued  or  the  decision  is  made  not  to  issue  the 
warning.  The  latter  of  these  options  will  be  discussed  first. 

3.4  Decisions  Not  to  Warn 

There  are  24  documented  decisions  not  to  warn.  However,  only  20  of  these 
decisions  had  a  clearly  defined  category  of  activity.  Figure  3.4  illustrates  the  distribution 
of  categories  of  activities  for  decisions  not  to  warn.  Only  the  broad  categories  are 
included  because  the  documentation  is  not  detailed  enough  to  break  the  decisions  into 


45 


subcategories  and  there  are  not  enough  warnings  to  differentiate  statistically  one 
subcategory  from  another. 


Decisions  Not  to  Warn 


Figures  3.4.  The  broad  categories  of  activity  leading  to  a  decision  not  to  issue  a  warning. 
Table  3.1  defines  the  categories  of  activities.  Decisions  not  to  issue  a  total  of  17  severe 
thunderstorm  warnings  and  3  tornado  warnings  are  included.  The  numbers  above  the 
columns  are  the  column  totals  which  will  be  used  in  the  statistical  analysis  that  follows. 


Notice  also  that  when  all  warnings  are  considered,  12  of  the  20  decisions  not  to 
issue  warnings  are  triggered  by  ground  truth  reports.  These  decisions  are  usually  made 
when  the  storm  coordinator  knows  that  the  cell  is  directly  over  a  populated  area,  and  yet 
initiated  and  received  ground  truth  reports  do  not  indicate  severe  weather  is  occurring. 

From  the  statistics  provided  in  Table  3.4,  notice  that  Ho  is  rejected  for  severe 
thunderstorm  warnings.  The  strict  conclusion  is  that  Pj  is  not  equal  to  1/3  for  at  least  one 
of  the  categories  of  activities.  However,  from  the  data  illustrated  in  Fig.  3.4,  it  can  be 
inferred  that  decisions  not  to  issue  severe  thunderstorm  warnings  are  more  likely  to  be 
triggered  by  ground  truth  reports  or  reflectivity  based  products  than  velocity  based 
products. 
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Table  3.4.  The  contingency  table  results  for  the  decisions  not  to  warn.  The  test 
determines  the  differences  between  the  broad  categories  of  activity  for  decisions  not  to 
issue  SVR  and  TOR  warnings. 


Statistical  Results:  Decisions  Not  to  Warn 


n 

exp 

32  ^2 

^  crit 

Conclusion 

SVR 

broad  categories 

17 

0.333 

5.67 

9.29  5.99 

Reject  Ho 

TOR 

broad  categories 

3 

0.333 

1 

Insufficient  Sample  Size 

3.5  The  Trigger 

Except  for  the  decisions  not  to  warn,  the  only  other  passage  out  of  the 
“Investigate  the  Cell”  block  is  to  encounter  a  trigger.  Notice  also  from  Figure  3. 1  that  the 
radar  operator  can  also  reach  the  trigger  block  directly  from  monitoring  the  situation  or 
from  the  verification  of  a  previous  warning.  The  trigger  is  defined  as  that  which  causes 
the  radar  operator  to  make  the  decision  to  issue  a  warning.  Recall  that  sample 
independence  is  a  fundamental  assumption  for  using  a  contingency  table  and  therefore 
each  trigger  is  categorized  by  only  one  category  of  activity  from  Table  3.1.  In  the 
SWATR,  the  rationale  for  some  warnings  is  clearly  stated  and  thus  the  trigger  is  easily 
pinpointed.  If  more  than  one  category  of  activity  is  listed  as  the  rationale  for  issuing  the 
warning,  the  last  documented  activity  is  used  as  the  trigger.  Similarly,  in  cases  where  the 
reasoning  for  the  issuance  of  a  warning  is  not  explicitly  stated,  the  trigger  is  taken  to  be 
the  last  documented  activity  prior  to  the  issuing  of  the  warning.  The  following  figures 
show  the  distribution  of  triggers. 
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Triggers:  Subcategories 


Figure  3.5.  The  triggers  for  issuing  severe  thunderstorm  (SVR)  and  tornado  (TOR) 
warnings.  Figure  3.5A  uses  the  broad  categories  while  Fig.  3.5B  utilizes  the 
subcategories  of  triggers.  Table  3.1  defines  the  categories  of  activities.  A  total  of  46 
severe  thunderstorm  warnings  and  20  tornado  warnings  are  included.  The  numbers 
above  the  columns  are  the  column  totals  which  will  be  used  in  the  statistical  analysis  that 
follows. 


Three  statistically  significant  conclusions  can  be  about  the  warning  triggers  as 
shown  in  Table  3.5. 


Table  3.5.  The  statistical  results  for  the  trigger.  The  test  determines  the  differences 
between  the  broad  and  subcategories  of  triggers  for  SVR  and  TOR  warnings. 
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Statistical  Results; 

The  Trigger 

n  Pi 

exp 

■*'  crit 

Conclusion 

SVR 

broad  categories 

46  0.333 

15.333 

15.826 

5.99 

Reject  Ho 

subcategories 

46  0.125 

5.75 

33.304 

14.1 

Reject  Ho 

TOR 

broad  categories 

20  0.333 

6.667 

7.6 

5.99 

Reject  Ho 

subcategories 

20  0.125 

2.5 

Insufficient  Sample  Size 

From  the  rejection  of  Hq  for  the  broad  categories  of  severe  thunderstorm  triggers,  it  can 
be  inferred  that  for  a  greater  number  of  severe  thunderstorm  warnings  are  triggered  by 
reflectivity  based  radar  products  than  velocity  products.  The  null  hypothesis  is  also 
rejected  for  the  subcategories  of  triggers  for  severe  thunderstorm  warnings.  However,  in 
this  case  there  is  no  clear  inference  and  therefore  the  only  conclusion  that  can  be  made  is 
that  Pj  is  not  equal  to  1/8  for  at  least  one  of  the  subcategories.  Finally,  with  the  broad 
categories  of  triggers  for  tornado  warnings,  it  can  be  concluded  that  the  trigger  is  more 
likely  to  be  velocity  based  than  reflectivity  based. 

3.6  Immediate  Trigger  Warnings 

A  subset  of  the  warnings  entering  the  “trigger”  block  do  not  come  from  the 
investigation  of  a  cell.  These  warnings  are  the  immediate  trigger  warnings  and  eleven 
come  directly  from  the  “monitor  the  situation”  block  while  ten  are  from  the  “verify 
warning”  box.  These  21  documented  immediate  trigger  warnings  are  all  severe 
thunderstorm  warnings.  Therefore,  for  the  PDS,  there  was  an  initiator  and  an 
investigation  of  the  cell  prior  to  all  20  tornado  warnings.  This  indicates  that  the  radar 
operator  is  more  hesitant  to  recommend  that  a  tornado  warning  should  be  issued  and 
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he/she  takes  time  to  investigate  the  cell.  Figure  3.6  shows  the  breakdown  of  trigger 
categories  for  the  immediate  trigger  warnings. 


Figure  3.6.  The  triggers  for  immediate  trigger  warnings.  Figure  3.6A  uses  the  broad 
categories  while  3.6B  utilizes  the  subcategories  of  triggers.  Table  3.1  defines  the 
categories.  All  2 1  immediate  trigger  warnings  are  severe  thunderstorm  warnings.  The 
numbers  are  the  column  totals,  which  will  be  used  in  the  statistical  analysis  that  follows. 


Immediate  trigger  warnings  are  a  subset  of  all  triggers  which  were  presented  in  section 
3.5.  Notice  that  48  percent  of  the  immediate  trigger  warnings  are  triggered  by  ground 
truth  reports.  However,  from  Fig.  3.5  only  33  percent  of  the  triggers  for  all  severe 
thunderstorm  warnings  are  ground  truth  reports.  Now  comparing  Figs.  3.5  to  Fig.  3.6, 


there  are  16  total  severe  thunderstorm  warnings  triggered  by  ground  truth  reports,  and  10 
of  these  16  were  immediate  trigger  warnings.  This  indicates  that  while  groimd  truth 
reports  are  not  the  most  common  trigger  for  severe  thunderstorm  warnings,  they  often 
cause  warnings  to  be  immediately  issued.  Table  3.6  presents  the  statistical  results. 

Table  3.6.  The  statistical  results  for  immediate  trigger  warnings.  The  test  determines  the 
differences  between  the  broad  and  subcategories  of  triggers  for  SVR  warnings.  There  are 
no  immediate  trigger  tornado  warnings  in  the  PDS. 

Statistical  Results:  Immediate  Trigger  Warnings 

n  P,  exp  j?  I^crit  Conclusion 

SVR 

broad  categories  21  0.333  7  7.71  5.99  Reject  Ho 

subcategories  21  0.125  2.625  Insufficient  Sample  Size 

From  the  rejection  of  Ho  for  the  broad  categories  of  triggers,  it  can  be  concluded  that 
ground  truth  reports  and  reflectivity  based  products  are  the  most  common  immediate 
triggers  for  more  severe  thunderstorm  warnings.  Notice  that  the  sample  size  was  too 
small  to  analyze  the  subcategories  of  activities. 

3.7  Issue  the  Warning 

The  only  exit  from  the  “trigger”  block  is  to  issue  the  warning.  An  analysis  of  the 
warning  preparation  time  will  now  be  presented.  The  preparation  time  is  defined  as  the 
time  elapsed  from  the  decision  to  issue  the  warning,  the  trigger,  to  the  issuance  of  the 
warning.  Figure  3.7  shows  the  distribution  of  preparation  times  for  warnings. 
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Warning  Preparation  Time 


Time  (min.) 

Figure  3.7.  The  warning  preparation  time  for  the  PDS  for  severe  thunderstorm  and 
tornado  warnings. 

The  average  preparation  time  for  a  warning  was  2. 1  minutes  with  a  median  of  two 
minutes  and  a  mode  of  one  minute.  Notice  that  all  documented  warnings  were 
transmitted  within  six  minutes  of  the  decision  to  issue  the  warning. 

The  warnings  with  the  shortest  preparation  times  were  likely  prepared  during  the 
investigation  of  the  cell  and  were  ready  to  be  transmitted  as  soon  as  a  trigger  occurred. 
However,  there  was  incomplete  documentation  as  to  which  warnings  were  prepared 
ahead  of  time,  and  thus  this  case  cannot  be  studied  independently. 

In  contrast,  there  are  several  reasons  for  a  long  preparation  time.  On  a  number  of 
occasions  the  forecast  office  had  trouble  with  the  software  used  to  type  the  warnings. 
Also,  increased  detail  in  the  warning  text  lengthens  the  preparation  time.  In  some 
circumstances  it  is  imperative  that  the  warning  be  immediately  transmitted,  but  in  other 
cases  a  detailed  warning  is  more  appropriate.  Thus  in  the  latter  situations,  a  longer 
preparation  time  seems  j  ustifiable. 
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3.8  Call  for  Ground  Truth 


After  the  issuing  of  a  warning,  the  forecast  office  generally  seeks  ground  truth 
reports.  For  this  analysis,  the  SDS  will  be  used  because  the  SWATR  are  not  needed. 
However,  two  episodes  have  incomplete  action  logs  and  therefore  only  332  of  the  351 
warnings  will  be  considered.  It  should  also  be  noted  that  ham  radio  reports  will  be 
included  with  the  phone  calls  as  sources  of  ground  truth  reports.  Finally,  unless 
otherwise  stated,  both  initiated  and  received  calls  are  considered  together.  It  is  often 
difficult  to  separate  the  two  because  the  WFO  will  not  initiate  a  call  if  one  has  already 
been  received.  Figure  3.8  shows  the  number  of  calls  to  each  of  the  warned  counties. 


Number  of  Calls  per  Warning 


Number  of  Calls 


Figure  3.8.  The  Number  of  initiated  and  received  calls  per  warning  for  the  SDS 
excluding  19  warnings  with  incomplete  Action  Logs. 


Notice  that  58  percent  of  the  warned  counties  are  called  once  or  twice  but  15 
percent  are  not  called.  There  are  several  possible  explanations  for  the  cases  where  the 
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county  was  not  called  during  the  severe  weather  episode.  First,  during  episodes  with 
multiple  valid  warnings,  it  is  difficult  to  keep  track  of  all  the  warned  counties.  Secondly, 
there  may  have  been  calls  that  were  never  recorded  on  an  Action  Log.  Again,  this  would 
be  more  likely  to  happen  during  an  intense  episode  with  many  calls.  Here,  a  report  of 
conditions  not  meeting  severe  weather  criteria  could  be  lost. 

3.9  Verify  the  Warning 

Calls  with  ground  truth  reports  are  the  only  way  the  forecast  office  can  receive 
real-time  verification  on  a  warning.  A  severe  call  is  defined  as  a  call  that  reports  a  severe 
local  storm  event.  Figure  3.9  illustrates  the  times  that  severe  calls  occur. 


Figure  3.9.  The  time  when  episode  verification  is  obtained  in  minutes  after  the  warning 
is  issued. 


There  are  severe  calls  for  126  warnings  which  equates  to  38  percent.  Over  70  percent  of 
the  severe  calls  occur  within  30  minutes  after  the  warning  is  issued.  The  severe  call  was 
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initiated  in  46  cases  while  it  was  received  68  times.  However,  it  should  be  noted  that  of 
the  68  times  the  severe  call  was  received,  33  of  the  coimties  had  an  initiated  call  prior  to 
the  received  severe  weather  report. 

Severe  calls  play  a  critical  role  in  verifying  warnings.  There  are  three  types  of 
verification  discussed  in  this  research:  real-time  verification,  episode  verification  and 
Storm  Data  verification.  ‘Real-time  verification’  is  defined  as  apparent  verification  of  a 
warning  from  a  severe  call  during  the  valid  warning  period.  Real-time  verification  aids 
the  forecaster  in  the  warning  process  by  providing  a  current  ground  truth  report  indicating 
a  cell  is  severe.  ‘Episode  verification’  encompasses  real-time  verification  as  well  as  any 
delayed  severe  calls  received  after  the  warning  has  expired  but  during  the  severe  weather 
episode.  A  report  that  the  weather  was  severe  an  hour  ago  during  a  valid  warning  period 
usually  does  not  aid  the  forecaster  in  the  warning  process  but  may  provide  input  to  Storm 
Data.  Finally,  'Storm  Data  verification’  is  based  on  a  Storm  Data  entry  indicating  that  a 
severe  local  storm  event  occurred  during  the  valid  warning  period  as  explained  in  chapter 
one.  Storm  Data  verification  will  be  described  in  detail  in  chapter  four.  For  all  three 
verification  schemes,  the  verification  rate  is  defined  as  the  percentage  of  warnings  that 
verify. 

First,  one  could  ask  if  taking  time  to  investigate  a  cell  increases  the  verification 
rate  To  answer  this  question,  the  real-time  verification  rate  was  analyzed  for  warnings 
with  both  an  initiator  and  a  trigger  as  well  as  for  immediate  trigger  warnings  for  the  PDS. 
Notice  from  Figure  3.1  that  47  warnings  have  both  an  initiator  and  a  trigger.  Of  these  47 
warnings,  25  had  real-time  verification  equating  to  a  53  percent  verification  rate.  Now 
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for  immediate  trigger  warnings,  8  of  21  warnings  had  real-time  verification  equating  to  a 
38  percent  verification  rate.  However,  using  a  contingency  table  analysis,  warnings  with 
both  an  initiator  and  a  trigger  do  not  have  statistically  higher  verification  rates  than 
immediate  trigger  warnings. 

Next,  a  comparison  was  made  between  the  real-time  verification  rates  for  first 
and  last  warnings  versus  alt  other  warnings.  All  the  analysis  for  the  remainder  of  this 
report  is  based  on  the  SDS.  The  hypothesis  was  that  first  warnings  would  have  lower 
verification  rates  because  ground  truth  reports  would  not  yet  be  available.  A  problem 
with  this  analysis  is  that  the  hypothesis  should  hold  for  the  first  warning  on  each  cell,  and 
during  active  episodes  there  are  multiple  cells.  However,  the  best  approximation 
available  is  the  first  warning  during  each  severe  weather  episode.  The  real-time 
verification  rate  for  first  warnings  was  0.27  as  compared  to  0.43  for  all  other  warnings. 
Using  a  contingency  table  analysis,  =  5.208,  and  therefore  the  verification  rate  is 
statistically  significantly  lower  for  first  warnings. 

Similarly,  last  warnings  should  have  lower  real-time  verification  rates  because  the 
forecast  office  might  have  difficulty  determining  when  to  stop  warning  for  a  cell.  Again, 
this  analysis  should  be  done  cell  by  cell  but  the  best  available  approximation  is  the  last 
warning  in  each  episode.  However,  there  was  not  a  statistically  significant  difference  in 
real-time  verification  rates  between  last  warnings  and  all  other  warnings. 

Sinee  the  national  verification  statistics  are  based  on  Storm  Data  verification,  it  is 
now  important  to  compare  episode  verification  to  Storm  Data  verification  as  shown  in 
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Table  3.7.  Episode  verification  is  used  rather  than  real-time  verification  in  order  to 
assess  the  accuracy  of  severe  calls. 


Table  3.7.  Storm  Data  and  episode  verification  (defined  in  the  text)  are  compared  using 
a  contingency  table.  The  data  set  is  the  SDS  excluding  19  warnings  with  missing  Action 
Logs. 


Episode  Verification 


Storm  Data  Verification 


Yes 

No 

Total 

Yes 

88 

38 

126 

No 

41 

165 

206 

Total 

129 

203 

332 

Looking  at  the  first  row,  notice  that  88  warnings  had  both  types  of  verification 
while  38  warnings  had  episode  but  not  Storm  Data  verification.  This  means  that 
88 

- =.70  of  the  warnings  with  episode  verification  appear  in  Storm  Data.  This  can  be 

126 

equated  to  the  accuracy  of  reports  during  the  episode  if  Storm  Data  is  accepted  as  the 
“truth”.  Another  way  consider  this  is  that  for  30  percent  of  the  warnings  with  episode 
verification,  the  radar  operator  thinks  he  or  she  has  verification  on  a  warning  while  the 
reports  are  actually  “false”. 

Now  considering  the  second  row,  there  are  41  warnings  that  have  Storm  Data 

41 

verification  but  do  not  have  episode  verification.  This  equates  to  - =.20  of  the 

206 

warnings  that  the  radar  operator  does  not  think  have  verified  will  have  severe  weather 
reports  at  a  later  time  and  become  a  part  of  Storm  Data.  Finally,  there  are  165  warnings 
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that  do  not  have  either  Storm  Data  or  episode  verification  which  encompasses  a  50 
percent  of  the  total  warnings. 
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4.  VERIFICATION  RESULTS 


Chapter  three  presented  the  importance  of  real  time  verification  as  a  forecast  tool 
as  well  as  a  comparison  between  episode  and  Storm  Data  verification.  This  chapter  will 
focus  on  Storm  Data  verification  by  first  presenting  the  past  and  present  verification 
statistics  and  then  providing  a  discussion  of  some  of  the  factors  affecting  the  statistics. 
Throughout  this  chapter,  the  SDS  is  used. 


4.1  Verification  Statistics 

To  put  the  RDU  WFO  verification  statistics  in  perspective,  the  national  statistics 
are  first  be  presented  in  Fig.  4. 1 . 


Figure  4.1.  The  national  verification  statistics  from  1984  to  1994  (Crowther  and 
Halmstad,  1995). 


Notice  the  general  improvement  of  the  verification  statistics  over  the  ten  year  period. 
However,  it  can  be  seen  that  the  FAR  increases  slightly  over  the  last  two  years.  However, 
Fig.  4. 1  does  not  separate  the  stations  with  the  WSR-88D  from  those  that  were  still  using 
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the  conventional  radar.  The  national  verification  statistics  for  stations  with  the  WSR- 
88D  from  March  1,  1991  to  June  30,  1995  are  as  follows;  POD  =  0.80,  FAR  =  0.45  and 
CSI  =  0.48  (Burgess,  1996). 

Now  focusing  on  the  local  scale.  Fig.  4.2  illustrates  the  verification  statistics  for 
the  RDU  WFO  from  1984  to  1994.  The  explanation  for  the  trends  prior  to  this  analysis 
period  is  unknown  but  clearly  there  was  an  increase  in  the  FAR  in  1988  and  a 
corresponding  decrease  in  the  CSI.  Also,  the  POD  was  low  in  1991. 


Verification  Statistics  for  the  RDU  WFO: 
1984-1994 


Figure  4.2.  The  verification  statistics  for  the  RDU  WFO  from  1984-1994  (Crowther  and 
Halmstad,  1993-1995,  Grenier  and  Halmstad,  1987  and  1991-1992,  Grenier,  Halmstad 
and  Lefitwich,  1988-1990,  and  Leftwich  and  Grenier,  1985-1986). 


The  above  figure  again  does  not  indicate  the  warnings  after  the  RDU  WFO  used  the 
WSR-88D  and  therefore  Table  4. 1  is  included.  Notice  that  the  table  also  separates  the 
1994  and  1995  statistics  for  the  SDS. 


60 


Table  4.1.  Verification  statistics  for  the  RDU  WFO  before  and  after  the  operational  use 
of  the  WSR-88D.  Note  that  the  data  set  for  pre  WSR-88D  is  1992-1993  and  the  post 
WSR-88D  statistics  are  taken  from  the  SDS.  The  last  two  columns  are  a  breakdown  of 
the  SDS  into  the  1994  and  1995  warnings. 


RDU  WFO  Verification  Statistics: 

Pre  WSR-88D 
(1992-1993) 

Post  WSR-88D 
(SDS) 

1994 

(SDS) 

1995 

(SDS) 

Verified  Warnings  (X) 

209 

142 

80 

62 

Misses  (Y) 

121 

52 

24 

28 

Unverified  Warnings  (Z) 

166 

209 

64 

145 

Total  Warnings  Issued 

375 

351 

144 

207 

POD 

0.63 

0.73 

0.77 

0.69 

FAR 

0.44 

0.59 

0.44 

0.7 

CSI 

0.42 

0.35 

0.48 

0.26 

The  next  step  is  to  examine  the  statistical  contrasts  between  the  periods  using  a  Z  test  for 
the  difference  between  two  population  proportions.  The  pre  and  post  WSR-88D  POD 
will  be  compared  as  an  example  of  this  test.  A  higher  POD  was  hypothesized  with  the 
development  of  the  WSR-88D  and  therefore  a  one  tailed  test  will  be  used.  The  null  and 
alternate  hypotheses  are: 


Ho!  (PODpost  -  PODpre)  -  0 

Ha:  (PODpost  -  PODpre)  >  0  (Mendenhall  and  Sincich,  1992). 


The  test  statistic  is:  Z  = 


POD^-POD^^ 

post  pre 


lx  , 

+  X 

r 

+  X 

>1 

r  1 

1 1 

I  post 

pre 

1- 

post 

pre 

V 

+  n 

pre 

+  n 

pre 

y 

post 

^pre) 

where  n;  =  X;  +  Y;,  the  denominator  of  the  proportion  being  tested  (Mendenhall  and 
Sincich,  1992).  The  rejection  region  is  |Z|  >  Zcrit  for  a  one  tailed  test.  At  the  0.05 
confidence  level,  Zcrit  is  1.645.  This  Zcrit  will  be  used  throughout  the  remainder  of  this 
paper  unless  otherwise  specified.  In  this  example,  Z  =  2.32  and  hence  Ho  can  be  rejected. 
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Therefore,  the  POD  is  statistically  significantly  higher  for  the  RDU  WFO  after  the 
operational  use  of  the  WSR-88D  began  in  March  of  1994  over  the  previous  two  years. 

The  final  step  to  completing  the  Z  test  is  to  verify  that  the  sample  is  “sufficiently 
large”.  Some  authors  suggest  looking  at  the  following  intervals  : 


POD  ±  2 


POD ^  (I -POD) 


for  both  the  pre  and  post  WSR-88D  periods  (Mendenhall 


and  Sincich,  1992).  If  neither  interval  contains  zero  or  one,  the  sample  is  sufficiently 
large.  For  this  example,  the  intervals  are  from  0.58  to  0.69  for  the  pre  WSR-88D  period 
and  from  0.67  to  0.80  for  the  post  WSR-88D  sample.  Because  neither  of  these  intervals 
contains  zero  or  one,  the  assumption  of  a  large  sample  is  met.  For  the  remainder  of  this 
chapter,  unless  otherwise  stated,  the  assumption  of  a  large  sample  has  been  met  and  will 
not  be  presented. 

There  are  only  two  changes  to  the  Z  test  for  two  tailed  cases.  First,  for  the  above 
example  the  alternate  hypothesis,  Ha,  would  state  that  (PODpost  -  PODpre)  ^  0. 
Secondly,  and  the  rejection  region  at  a  confidence  level  of  .05  has  .025  in  each  tail  and 
hence  Zcrit=  ±  1.96. 


Table  4.2.  The  resulting  Z  test  statistics  for  the  difference  between  two  population 
proportions  for  the  POD,  FAR  and  CSI.  The  test  was  performed  between  Pre  and  Post 
WSR-88D  periods  and  for  the  1994  and  1995  SDS.  The  statistics  and  data  sets  are  given 
in  Table  4.1. 


Z-Test  Statistics 

POD 

FAR 

CSI 

Pre  vs  Post 
WSR-88D 

-2.32 

-4.12 

2.11 

1994  vs  1995 

1.25 

-4.81 

4.4 
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Table  4.2  shows  that  is  also  rejected  for  the  FAR  and  CSI  using  either  a  one  or  two 
tailed  test  in  the  pre  versus  post  WSR-88D  comparison.  However,  from  the  data 
provided  in  table  4.1,  it  can  thus  be  inferred  that  the  FAR  and  CSI  have  gotten  worse 
since  the  WSR-88D  became  operational  at  the  RDU  WFO.  Looking  at  the  1994  versus 
1995  comparison.  Ho  is  not  rejected  for  the  POD  but  is  rejected  for  the  FAR  and  CSI. 
Again,  it  can  be  inferred  that  the  FAR  and  the  CSI  have  gotten  worse. 

For  both  the  pre  versus  post  WSR-88D  comparison  as  well  as  the  1994  versus 
1995  test,  note  that  the  increasing  FAR  drives  the  decreasing  CSI.  This  is  because  the 
POD  is  increasing  or  remaining  constant  for  the  cases  presented  in  this  section,  and  the 
CSI  is  directly  proportional  to  the  POD.  Thus  only  the  rationale  for  the  increasing  FAR 
will  be  presented.  Finally,  it  should  be  pointed  out  that  there  was  also  a  slight  increase  in 
the  national  FAR  from  1992  to  1994  as  shown  in  Figure  4.1. 

A  hypothesis  for  the  increasing  FAR  is  that  there  is  a  tendency  to  over  warn  based 
on  WSR-88D  products.  When  operational  use  of  the  WSR-88D  began  in  1994,  only  a 
few  radar  operators  at  the  RDU  WFO  had  been  trained  on  the  Doppler  radar  .  Therefore, 
each  of  the  trained  operators  worked  numerous  severe  weather  episodes  and  gained  a  lot 
of  warning  experience  during  the  1994  severe  weather  season.  However,  many  more 
forecasters  were  trained  on  the  Doppler  radar  by  the  1995  severe  weather  season.  This 
meant  that  numerous  radar  operators  were  having  their  first  operational  WSR-88D 
experiences  during  the  1995  severe  weather  season.  There  is  a  steep  learning  curve  for 
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the  WSR-88D  as  discussed  in  section  1.4,  and  therefore  the  verification  statistics  could 
be  expected  to  improve  in  future  years  if  this  hypothesis  is  true. 

4.2  Factors  Affecting  Verification  Statistics 

The  previous  segment  presented  the  verification  statistics  for  the  RDU  WFO  and 
this  section  will  explore  some  of  the  factors  affecting  these  verification  statistics. 
Because  the  1994  and  1995  FAR  and  CSI  were  shown  to  be  statistically  significantly 
different,  these  verification  statistics  for  the  two  year  groups  will  be  examined  separately 
when  each  category  has  a  sufficiently  large  number  of  warnings.  However,  the  POD  may 
be  examined  for  the  entire  SDS  because  there  where  not  statistically  significant 
differences  found  in  the  previous  section. 

4.2.1  Storm  Environment 

Clearly,  the  storm  environment  has  an  important  impact  on  the  verification 
statistics.  However,  there  is  not  consistent  documentation  in  the  SWATR  of  the 
environment  surrounding  the  severe  weather  episodes.  For  example,  if  the  stability 
indices  had  been  recorded  for  all  the  cases,  episodes  with  similar  indices  could  perhaps 
be  grouped  together  and  analyzed.  However,  even  if  such  documentation  were  available, 
it  is  questionable  whether  or  not  the  warnings  would  fall  into  clear  categories.  Therefore, 
two  arbitrary  breakdowns  of  the  SDS  have  been  made:  (1)  episodes  with  severe 
thunderstorm  or  tornado  watches  versus  episodes  without  watches  and  (2)  episodes  with 
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more  than  four  warnings  (defined  here  as  outbreaks)  versus  those  with  four  or  less 
warnings  (non-outbreaks). 

First,  the  watch  versus  no  watch  periods  will  be  covered  as  summarized  in  the 
table  4.3. 


Table  4.3.  This  table  summarizes  the  verification  statistics  and  Z-tests  used  in  comparing 
watch  and  no  watch  periods.  The  Z  test  statistic  is  listed  in  the  first  column  of  the  two 
categories  being  compared  and  bold  numbers  represent  cases  where  Ho  may  be  rejected. 


Watch  Versus  No  Watch  Periods 


Watch 

No  Watch 

1995:  Watch 

1995: 

No  Watch 

Verified 

Warnings 

(X) 

105 

137 

59 

21 

46 

16 

31 

21 

15 

22 

6 

Misses 

(Y) 

Unverified 

Warnings 

(Z) 

140 

69 

43 

21 

97 

48 

Total  Warnings 

245 

106 

102 

42 

143 

64 

Issued 

POD 

0.77 

0.64 

0.87 

0.58 

0.68 

0.73 

Z 

1  1.93 

3.27 

-0.45 

FAR 

0.57 

0.65 

0.42 

0.50 

0.68 

0.75 

Z 

-1.39 

-0.86 

-1.05 

CSI 

0.38 

0.29 

0.53 

0.37 

0.28 

0.23 

z 

1.74 

2.00 

0.80 

Notice  that  the  POD  is  statistically  significantly  higher  during  watch  periods  than  no 
watch  periods.  However,  there  are  no  differences  in  the  FAR  for  any  of  the  watch/no 
watch  breakdowns.  Finally,  the  CSI  was  higher  for  the  1994  warnings  in  the  SDS  during 
watches  than  no  watch  periods  while  there  is  no  difference  in  the  CSI  for  the  1995 


period. 


Secondly,  outbreak  versus  non-outbreak  periods  will  be  compared  as  summarized 
in  Table  4.4.  The  conclusions  made  for  the  previous  watch/no  watch  analysis  hold  for 
the  outbreak/non-outbreak  comparison  with  the  contrasts  magnified. 


Table  4.4.  This  table  summarizes  the  verification  statistics  and  Z-tests  used  in  comparing 
outbreak  and  non-outbreak  periods.  The  Z  test  statistic  is  listed  in  the  first  column  of  the 
two  categories  being  compared  and  bold  numbers  represent  cases  where  Hq  may  be 
rejected. 

Outbreaks  Versus  Non-outbreaks 


Outbreaks 

Non- 

1994 

1994  Non- 

1995 

1995  Non- 

Outbreaks 

Outbreaks 

Outbreaks 

Outbreaks 

Outbreaks 

Verified 

Warnings  (X) 

115 

27 

68 

8 

47 

11 

24 

28 

18 

18 

10 

Misses  (Y) 

Unverified 
Warnings  (Z) 

164 

45 

52 

16 

116 

33 

Total  Warnings 
Issued 

279 

72 

120 

24 

163 

44 

POD 

0.83 

0.49 

0.30 

0.52 

Z 

4.77 

6.28 

1.69 

FAR 

0.59 

0.63 

0.43 

0.67 

0.71 

0.75 

Z 

-0.57  1 

-2.09 

-0.50 

CSI 

0.38 

0.27 

0.54 

0.19 

0.26 

0.20 

z 

1.99 

3.94 

0.84 

The  POD  is  again  statistically  significantly  higher  for  outbreaks  than  non¬ 
outbreaks.  Notice  the  dramatic  contrasts  in  the  POD  especially  during  1994  with  the 
outbreak  category  having  0.91  and  only  0.30  for  non-outbreaks.  Further,  the  1994 
improvement  is  not  at  the  expense  of  a  higher  false  alarm  rate.  The  FAR  is  in  fact  lower 
for  1994  outbreaks.  Therefore,  the  CSI  is  also  enhanced  for  1994.  There  are  no 
differences  in  the  FAR  and  CSI  between  outbreak  and  non-outbreak  periods  for  1995. 
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The  conclusion  of  a  higher  POD  during  watch  periods  and  outbreaks  is  not  a 
surprising  result.  Watch  period  and  outbreaks  are  often  associated  with  large  storm 
systems.  “Giant”  storms  generally  have  better  verification  statistics;  during  the  JDOP 
there  were  many  large  storms,  and  hence  the  resulting  verification  statistics  may  have 
been  too  encouraging  (Burgess,  1996). 

4.2.2  The  Verification  Process 

Obviously,  the  verification  statistics  are  affected  by  the  procedures  used  to  derive 
them.  This  section  will  highlight  two  characteristics  of  the  current  national  verification 
process:  double  jeopardy  warning  and  warned  counties. 

First  an  example  will  be  presented  to  illustrate  the  case  of  double  jeopardy 
warnings.  Suppose  a  Storm  Data  entry  is  recorded  at  10:00,  the  warning  was  issued  at 
10:05  and  no  fiirther  Storm  Data  entries  were  made.  Then  both  an  xmverified  warning 
(Z)  and  a  miss  (Y)  are  recorded.  The  same  holds  true  when  the  Storm  Data  entry  appears 
after  a  warning  expires.  Both  of  these  cases  will  be  called  ‘double  jeopardy’  since  two 
penalties  are  given. 

In  the  SDS,  there  are  16  cases  of  double  jeopardy.  Half  of  these  cases  have  the 
warning  within  ten  minutes  after  the  Storm  Data  entry  while  the  remaining  eight  have  the 
Storm  Data  entry  within  ten  minutes  after  the  warning  expires.  Assuming  these  double 
jeopardy  cases  verified  and  there  was  no  miss,  the  POD  and  CSI  increase  by  .02  and  .04 
respectively  while  the  FAR  decreases  by  .05.  Clearly,  crediting  the  forecast  office  with 
both  a  verified  warning  and  no  miss  would  allow  the  most  enhancements  in  the 


67 


verification  statistics,  and  yet  the  improvements  are  small.  If  either  a  verified  warning  or 
no  miss  were  recorded,  less  significant  enhancements  in  the  verification  statistics  would 
be  seen.  Thus,  double  jeopardy  warnings  do  not  have  a  very  significant  effect  on  the 
verification  statistics. 

The  second  characteristic  of  the  current  warning  process  is  that  each  warned 
county  is  verified  independently.  This  means  that  when  a  warning  was  issued  for  two 
counties  for  the  same  cell,  each  county  is  verified  independently.  This  circumstance 
when  warnings  are  issued  for  more  than  one  county  be  called  multiple  warnings.  A 
reasonable  hypothesis  is  that  verification  would  only  be  received  for  one  of  the  warned 
counties  in  the  multiple  warning  case,  and  hence  the  verification  rates  would  be  lower  for 
multiple  warnings.  For  the  SDS,  the  verification  rate  was  .35  for  multiple  warnings 
versus  .44  for  individually  warned  counties.  Using  a  contingency  table,  X  =3.215 
indicating  that  there  is  not  a  statistically  significant  difference  in  verification  rates  for  the 
two  classes  of  warnings. 

4.2.3  Time  of  Day 

The  time  of  day  impacts  the  number  of  warnings  issued  as  well  as  the  verification 
rate  as  shown  in  Fig.  4.3.  Note  that  the  verification  rate  is  the  only  statistic  analyzed  here 
because  the  data  set  was  too  small  to  break  up  the  misses  (Y)  into  6  categories.  Also,  the 
SDS  has  not  been  partitioned  by  year  for  this  analysis  because  of  the  necessity  of  having 
a  sufficiently  large  sample  size. 
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It  is  not  remarkable  that  there  are  more  warnings  issued  in  the  late  afternoon  and 
evening  hours.  However,  notice  that  the  verification  rate  increases  steadily  throughout 
the  day.  Using  a  contingency  table  analysis,  X  =  17. 13  which  is  greater  than  I  crit ""  H  I 
with  five  degrees  of  freedom.  Therefore,  it  can  be  concluded  that  the  verification  rate 
increases  throughout  the  day.  The  conclusion  to  section  4.2. 1  states  that  larger  storms 


Figure  4.3.  The  number  of  warnings  issued  and  the  verification  rate  by  the  hour  of  the 
day.  The  SDS  was  used  for  this  analysis. 

have  higher  verification  rates.  Taking  this  rationale  one  step  further,  these  large  storms 
generally  occur  in  the  late  afternoon  and  evening  hours,  and  therefore  the  verification 
rate  is  higher  during  this  period.  Also,  the  verification  rate  is  lowest  during  the  night 
while  most  people  are  sleeping. 

4.2.4  Population  Density 

According  to  Hales  and  Kelly  (1985),  national  data  from  1979  to  1985  showed 
that  significantly  more  warnings  were  issued  to  highly  populated  regions.  To  see  if  this 
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trend  was  present  in  the  SDS,  regression  models  were  used  to  determine  the  effects  of 
population  density  on  both  the  verification  rate  and  the  number  of  warnings  issued.  Both 
regression  models  were  performed  using  SAS  and  the  programs  are  included  in  appendix 
7.4.  Since  warnings  are  issued  by  counties,  this  analysis  uses  the  county  population 
densities.  This  could  cause  errors  because  counties  are  not  homogeneously  populated. 

The  first  model  used  a  logistic  regression  to  relate  the  verification  rate,  the 
dependent  variable,  to  the  population  density,  the  independent  variable.  This  is  a  special 
class  of  regression  models  where  the  dependent  variable  is  bounded  between  zero  and 
one  and  the  errors  have  a  binomial  distribution  (Hosmer  and  Lemeshow,  1989).  The  I 
distribution  with  one  degree  of  freedom  is  used  to  determine  if  the  independent  variable 
is  important  to  the  model.  In  this  example,  =  1.18  which  is  not  greater  than  and 
hence  the  population  density  does  not  have  statistically  significant  impact  on  the  model. 

Because  the  population  density  did  not  have  an  important  effect  on  the 
verification  rate,  the  next  step  was  to  see  if  fewer  warnings  were  issued  to  the  sparsely 
populated  counties  because  of  the  problems  with  verification.  For  this  regression,  the 
number  of  warnings  was  the  dependent  variable  and  the  county  area  and  population 
density  were  the  independent  variables.  A  Poisson  regression  was  used  because  the 
dependent  variable  has  the  characteristics  of  a  Poisson  random  variable; 

1)  The  experiment  involves  counting  the  number  of  occurrences,  i.e.  the  number 
of  warnings  issued,  in  a  given  unit  of  measurement,  in  this  case  the  county. 

2)  The  number  of  occurrences  (warnings)  for  different  counties  are  independent 
(Mendenhall  and  Sincich,  1992). 
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The  values  for  this  case  were  15.30  for  the  counties  area  and  0.11  for  the  population 
density.  Therefore,  the  county’s  area  has  a  significant  impact  on  the  number  of  warnings 
issued  with  fewer  warnings  being  issued  to  smaller  counties.  However,  the  population 
density  did  not  have  a  statistically  significant  impact  on  the  number  of  warnings  issued. 
Because  county  area  was  a  significant  term  in  the  model  while  population  density  was 
not,  it  is  unlikely  that  the  insignificant  effect  of  population  density  is  due  to  the  sample 
being  too  small. 

4.2.5  Per  Capita  Income 

A  similar  analysis  was  conducted  to  determine  the  importance  of  the  per  capita 
income  on  the  verification  rate.  A  reasonable  hypothesis  was  that  a  higher  per  capita 
income  would  indicate  a  higher  education  level  and  hence  a  greater  awareness  that  the 
WFO  should  be  contacted  to  report  severe  local  storm  events.  The  same  two  tests  were 
conducted  as  described  in  the  previous  section.  For  the  logistic  regression,  the 
verification  rate  was  the  dependent  variable  with  the  per  capita  income  as  the 
independent  variable.  The  resulting  was  3.27  which  is  not  greater  that  I^crit  ai^d  hence 
the  per  capita  income  does  not  have  a  statistically  significant  impact  on  the  verification 
rate.  The  second  test  conducted  was  a  Poisson  regression  testing  the  impact  of  the 
population  density  and  per  capita  income  on  the  number  of  warnings  issued.  The 
resulting  X^  values  were  0.047  and  1.65  for  the  population  density  and  per  capita  income 
respectively.  Thus,  neither  of  the  independent  variables  had  a  significant  impact  on  the 
number  of  warnings  issued. 
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5.  CONCLUSIONS  AND  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 


Meteorologists  first  started  forecasting  severe  thunderstorms  and  tornadoes  in  the 
nineteenth  century  and  verification  procedures  were  developed  soon  after.  From  the 
1800’s  to  the  present,  researchers  have  been  trying  to  improve  the  severe  weather 
forecast  process.  In  recent  years,  the  Doppler  radar  was  developed  in  an  attempt  to 
increase  severe  weather  warning  timeliness  and  accuracy.  This  research  has  been  a 
preliminary  analysis  of  how  warnings  are  issued  using  the  WSR-88D  at  the  RDU  WFO. 

The  warning  process  was  described  in  terms  of  the  schematic  representation  of 
the  warning  process  presented  as  Fig.  3.1.  The  initiators,  triggers  and  decisions  not  to 
warn  are  categorized  by  either  the  radar  products  used  or  the  ground  truth  reports 
received. 

The  radar  operator  starts  by  monitoring  the  situation  throughout  the  county 
warning  area.  Then  something  happens,  the  initiator,  which  causes  the  radar  operator  to 
investigate  a  given  cell  and  consider  issuing  a  warning.  There  were  no  statistically 
significant  conclusions  as  to  which  initiators  occur  most  commonly.  However,  the 
initiator  leads  to  the  investigation  of  a  suspicious  cell.  The  average  duration  of  this 
investigation  is  7.2  minutes.  Then,  in  thirty  percent  of  the  cases  the  decision  is  made  not 
to  issue  the  warning.  Ground  truth  reports  and  reflectivity  based  products  are  equally 
important  in  these  decisions  not  to  warn  while  velocity  based  products  are  not  as 
commonly  used. 

For  the  remaining  seventy  percent  of  the  cases,  the  investigation  leads  to  a  trigger. 
A  trigger  is  defined  as  that  which  drives  the  decision  to  issue  the  warning.  The  trigger 


for  severe  thunderstorm  warnings  is  usually  reflectivity  while  for  tornado  warnings 
Doppler  velocity  products  are  most  commonly  used  immediately  prior  to  issuing  the 
warning. 

A  subset  of  the  warnings  (31  percent)  are  immediate  trigger  warnings.  These 
warnings  had  no  initiator  and  investigation  prior  to  the  issuance  of  the  waning.  All 
immediate  trigger  warnings  are  severe  thunderstorm  warnings.  There  is  a  dramatic 
increase  in  the  importance  of  ground  truth  reports  for  these  immediate  trigger  cases. 

The  trigger  leads  to  the  issuance  of  the  warning.  Ground  truth  reports  are  then 
sought  for  85  percent  of  the  warnings  to  both  to  aid  in  the  warning  process  as  well  as  to 
gain  verification  on  the  warnings  issued.  This  leads  to  the  verification  of  38  percent  of 
the  warnings  during  the  severe  weather  episode.  There  is  a  70  percent  accuracy  rate  on 
calls  reporting  severe  weather  if  Storm  Data  is  used  as  the  “truth”. 

Next,  Storm  Data  verification  was  used  to  analyze  the  verification  statistics.  The 
POD  at  the  RDU  WFO  has  improved  a  statistically  significant  amount  since  operational 
use  of  the  WSR-88D  began  in  March  of  1994.  However,  both  the  FAR  and  CSI  have 
gotten  worse  with  the  Doppler  radar.  In  the  period  with  the  WSR-88D,  the  POD  has  not 
changed  between  1994  and  1995  but  both  the  FAR  and  CSI  have  gotten  worse.  The  POD 
is  higher  during  periods  with  severe  thunderstorm  or  tornado  watches  as  well  as  for 
outbreaks.  The  verification  rate  increases  throughout  the  day.  Finally,  the  population 
density  and  per  capita  income  do  not  affect  either  the  verification  rate  or  the  number  of 
warnings  issued. 
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However,  the  reader  is  reminded  of  the  discussion  presented  in  section  1.2 
concerning  the  problems  with  using  verification  statistics  as  a  measure  of  warning 
accuracy.  Are  the  majority  of  the  severe  local  storm  events  that  occur  in  the  RWA 
reported?  Recall  that  the  OKC  WFO’s  POD  increased  from  .508  to  .729  and  the  FAR 
decreased  from  .801  to  .508  when  intensive  post  storm  surveys  were  initiated  in  1983  as 
shown  in  Table  1.2.  Would  increased  efforts  at  gaining  ground  truth  reports  have  similar 
effects  on  the  RDU  WFO’s  verification  statistics?  As  is  expected  elsewhere,  the  majority 
of  the  severe  weather  events  at  the  RDU  WFO  are  marginal  severe  local  storm  events 
such  as  dime  size  hail  or  wind  gusts  of  just  over  50  knots.  Do  such  events  merit  a  post 
storm  survey?  The  emphasis  should  perhaps  be  focused  on  the  warning  process  rather 
than  the  verification  statistics. 

This  thesis  serves  as  a  preliminary  examination  of  the  warning  process.  Future 
research  should  provide  more  specific  conclusions  about  the  process.  The  SWAT 
recorder  in  the  future  will  be  able  to  determine  the  validity  of  the  warning  process 
presented  in  this  report.  The  radar  operators  should  also  be  encouraged  to  verbalize  their 
thought  process  in  order  to  increase  the  accuracy  of  the  data.  In  addition,  it  would  assist 
the  documentation  if  the  identification  all  radar  products  presented  on  the  PUP  were 
automatically  recorded  along  with  the  time  when  presented.  This  would  allow  for  a  more 
detailed  statistical  analysis  of  the  WSR-88D  products  used  in  conjunction  with  successful 
and  unsuccessful  warnings.  A  final  recommendation  would  be  to  attempt  to  divide  the 
warnings  into  categories  in  order  to  determine  the  products  triggering  the  issuance  of 
warnings  for  hail,  straight  line  wind,  etc.  A  detailed  explanation  of  the  spread  sheet 
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organization  is  included  in  Appendix  7.2  to  assist  future  researches.  In  addition,  a 
computer  disk  containing  all  the  files  is  available  from  Dr.  Allen  J.  Riordan  at  NCSU. 
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7.  APPENDICES 
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7.1  Map  of  the  Raleigh  Warning  Area  (RWA) 
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The  interior  of  the  bold  enclosure  is  the  RWA.  The  dot  in  Johnston  County  near  the  Wake  County  line 
indicates  the  position  of  the  WSR-88D  in  Clayton,  NC. 


7.2  SPREADSHEET  ORGANIZATION 

The  data  has  been  organized  into  five  different  types  of  spreadsheets:  the  Episode 
Summaries,  the  Master  Warning  List,  the  Warning  Process  Spreadsheet,  the  Decisions 
Not  to  Warn,  and  the  List  of  Misses.  The  purpose  for  each  spreadsheet  will  be  briefly 
discussed  followed  by  a  detailed  explanation  of  each  column  entry.  Because  of  the 
excessive  length,  the  spreadsheets  have  not  been  included  in  this  report  and  are  available 
to  future  researchers  on  disk  from  Dr.  Allen  J.  Riordan  in  the  Department  of  Marine 
Earth  and  Atmospheric  Science  at  NCSU.  However,  the  spreadsheets  used  in  this  thesis 
are  by  no  means  the  best  manner  in  which  to  handle  the  data,  and  subsequent  researchers 

1 

! 

are  encouraged  to  devise  their  own  organization  method. 

7.2.1  Event  Summaries 

The  Episode  Summaries  compile  the  information  from  the  Warning  Logs,  Action 
Logs,  call  records  form  the  SWATR,  and  Storm  Data  by  episode  in  a  timeline  fashion  for 
the  ESDS.  This  phase  in  the  data  organization  process  is  simply  meant  to  put  the 
warning  and  call  information  together  in  an  usable  marmer  for  building  the  remaining 
spreadsheets.  The  column  entries  are  as  follows: 

A)  The  time  in  Eastern  Daylight  Time  (EDT)  (or  Eastern  Standard  Time  (EST)  if 

applicable)  using  a  24  hour  clock. 

B)  The  county  where  the  documented  activity  occurred. 

C)  The  location  within  the  county  (N,  S,  etc.).  This  designation  is  only  used  for 

warnings  issued  for  a  portion  of  the  county. 
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CALLS 


D  and  E)  Initiated  and  received  calls  are  classified  by  category.  ‘SVR’  and  ‘TOR’ 
categories  signify  calls  that  meet  severe  thunderstorm  or  tornado  criteria 
respectively.  Similarly,  ‘Not  SVR’  and  ‘Not  TOR’  represent  calls  that  fail  to 
meet  severe  thunderstorm  or  tornado  standards  as  defined  in  chapter  one.  Calls 
from  both  the  Action  Logs  and  the  SWATR  are  included.  The  time  recorded  in 
the  Action  Log  is  used  except  for  the  case  explained  for  column  F.  Most  calls 
listed  in  the  SWATR  are  also  in  the  Action  Logs  but  occasionally  a  call  was  never 
entered  in  the  Action  Log.  There  may  be  errors  in  the  analysis  of  ground  truth 
reports  because  of  missing  calls  in  the  Action  Log. 

F)  The  time  of  the  event  is  recorded  only  if  documented  in  the  Action  Logs  or  the 
SWATR.  For  other  cases,  the  only  information  known  about  the  time  of  the  event 
is  that  it  occurred  prior  to  the  call.  Unfortunately,  the  Action  Log  entries  are 
occasionally  listed  by  event  time  rather  than  call  time.  For  these  cases,  if  the 
SWATR  list  a  call  time,  the  SWATR  time  is  listed  in  column  A  and  the  Action 
Log  event  time  is  placed  in  column  F.  However,  SWATR  are  not  available  for  all 
episodes  and  their  focus  is  not  to  list  all  call  information.  There  may  be  cases 
when  the  Action  Log  lists  event  times  and  there  is  not  a  SWAT  Report  available 
to  correct  these  times  to  the  call  times.  If  the  event  time  is  listed  as  a  call  time, 
the  radar  operator  is  assumed  to  know  of  the  report  and  be  able  to  use  the 
information  in  the  decision  making  process. 
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G)  The  source  of  the  call  information  is  recorded  in  this  column  where  ‘A’  represents 

the  Action  Logs,  ‘S’  stands  for  the  SWATR,  and  ‘B’  is  for  both  the  Action  Log 
and  the  SWAT  Report. 

WARNINGS 

H)  All  severe  thunderstorm  (SVR)  and  tornado  (TOR)  warning  are  listed.  Also  MISS  is 

listed  for  misses  as  defined  in  chapter  one. 

I)  The  time  the  warning  expires— EDT  or  EST  using  a  24  hour  clock. 

J)  The  warning  number  during  the  given  severe  weather  episode.  For  cases  where  more 

than  one  county  was  warned,  all  counties  are  listed  with  the  same  warning 
number. 

K)  For  warning  verification,  a  Yes  or  No  is  listed  in  column  1 1  for  warnings  that  do  and 

do  not  verify  respectively. 

L)  Additional  information  explaining  entries  is  provided  in  this  column.  Examples  of 

entries  would  be  a  call  time  discrepancy  between  the  Action  Logs  and  the 
SWATR.  Column  L  also  gives  the  specific  information  about  how  warnings 
verified;  winds,  hail  (size),  or  a  tornado. 

7.2.2  Master  Warning  List 

The  Master  Warning  List  is  a  spreadsheet  v^th  351  rows  composed  of  the 
warning  in  the  ESDS  and  45  columns  containing  information  about  the  warning.  General 
episode  data  is  included  such  as  the  date,  total  number  of  warnings,  watch  information, 
and  if  the  SWAT  was  present.  The  remainder  of  the  spreadsheet  is  composed  of  specific 
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warning  information.  First,  the  time,  location,  warning  number  in  the  episode,  type  of 
warning  and  specific  verification  information  are  provided.  Then  a  number  of  columns 
list  variables  that  could  affect  verification  including:  if  the  episode  was  an  outbreak,  if 
the  warning  was  issued  during  sleeping  hours  of  2300-0700,  if  multiple  counties  were 
simultaneously  warned,  and  if  the  warning  was  a  double  jeopardy  warning.  Lastly, 
initiated  and  received  call  information  is  listed  for  before,  during  and  after  the  warning 
was  issued.  Next,  each  column  entry  will  be  explained.  Note  that  the  columns  that 
answer  questions  will  be  either  in  binary  (0  meaning  no  and  1  meaning  yes)  or  have  Y 
and  N  representing  yes  and  no  respectively. 

A-C)  The  warning  date,  year  and  time  (EDT  or  EST). 

D)  Was  the  SWAT  present  during  the  warning? 

E)  Was  the  SWAT  present  at  any  point  during  the  severe  weather  episode? 

F)  The  hour  of  the  day.  For  example,  a  warning  at  16:18  would  have  a  16  in  this 

column. 

G)  The  county  warned.  Note  that  each  warned  county  is  listed  separately  for  this 

spreadshee:t. 

H)  The  type  of  warning  issued— SVR  for  severe  thunderstorm  and  TOR  for  tornado. 

I)  The  warning  number  during  the  severe  weather  episode.  For  example,  the  third 

warning  of  the  episode  would  have  a  three.  The  warning  number  is  by  cell  and 
not  by  individual  warned  counties.  In  other  words  if  both  Wake  and  Durham 
Counties  were  warned  simultaneously,  both  warnings  would  have  the  same 
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warning  number.  Adjacent  counties  warned  simultaneously  are  assumed  to  be  for 
the  same  cell. 

J)  Were  warnings  issued  to  multiple  adjacent  counties  simultaneously? 

K)  Was  only  a  portion  of  the  county  warned  (N,  S,  etc.)? 

L)  Was  the  warning  issued  between  07;00  and  23;00? 

M)  Were  more  than  four  warnings  issued  during  the  severe  weather  episode?  In  other 

words,  was  the  episode  an  outbreak? 

N)  Was  there  a  valid  severe  thunderstorm  or  tornado  watch  for  anywhere  in  the  Raleigh 

Warning  Area? 

O)  Did  the  warning  verify? 

P-R)  Did  the  warning  verify  with  a  report  of  thunderstorm  winds,  hail  or  a  tornado? 

S)  Was  the  warning  a  case  of  double  jeopardy? 

T)  The  time  difference  between  when  the  warning  was  issued  (or  expired)  and  the  time 

of  the  storm  data  entry  in  ten  minute  increments  for  double  jeopardy  cases.  For 
example,  a  10  means  that  the  warning  expired  within  ten  minutes  of  the  storm 
data  entry.  Negative  times  indicate  that  the  warning  was  issued  after  the  storm 
data  entry  time. 

U)  Were  multiple  adjacent  counties  simultaneously  warned? 

V)  The  total  number  of  telephone  calls  (initiated  or  received)  and  ham  radio  reports 

were  recorded  from  the  warned  county. 
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W)  The  total  number  of  severe  thunderstorm  or  tornado  reports  (from  initiated  or 

received  telephone  calls  or  ham  radio  reports)  were  recorded  from  the  warned 
county. 

X)  The  total  number  of  received  telephone  calls  or  ham  radio  reports  reporting  severe 

thunderstorms  or  tornadoes. 

Y)  The  total  number  of  initiated  telephone  calls  reporting  severe  thunderstorms  or 

tornadoes. 

Z)  The  time  when  the  first  severe  thimderstorm  or  tornado  report  is  received  via  an 

initiated  or  received  telephone  call  or  a  ham  radio  report. 

AA)  Was  the  first  severe  weather  report  initiated  or  received? 

AB)  Was  there  an  initiated  call  prior  to  the  report  of  severe  weather?  If  the  first  severe 

weather  report  was  initiated,  the  answer  is  yes. 

AC-AG)  This  section  concerns  calls  prior  to  when  the  county  was  warned.  Column  AC 
gives  the  total  number  of  calls  prior  to  the  issuance  of  the  warning  and  the 
subsequent  columns  indicate  the  total  number  of  initiated  severe  weather  reports 
(AD),  initiated  reports  of  conditions  not  meeting  severe  weather  criteria  (AE), 
received  severe  weather  reports  (AF),  and  received  reports  of  conditions  not 
meeting  severe  weather  criteria  (AG). 

AH-AL)  This  section  concerns  calls  during  the  valid  warning  period  and  is  presented  in  a 
parallel  manner  to  the  previous  section. 

AM-AQ)  This  section  concerns  calls  after  the  warning  has  expired  and  is  presented  in  a 
parallel  marmer  to  the  previous  two  sections. 
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AR)  The  total  number  of  initiated  and  received  calls  during  the  severe  weather  episode 

to  the  warned  county.  In  cases  where  the  county  was  warned  more  than  once,  the 
calls  are  separated  between  the  two  warnings.  There  is  normally  a  logical  split. 
In  cases  where  the  warning  was  upgraded  from  a  severe  thunderstorm  to  a  tornado 
warning  and  both  warnings  are  valid,  the  calls  are  recorded  for  both  warnings. 

AS)  The  total  number  of  warnings  during  the  severe  weather  episode. 

7.2.3  The  Warning  Process  Spreadsheet 

The  Warning  Process  Spreadsheet  again  lists  the  warnings  by  rows  with  details 
concerning  the  warning  in  columns.  However,  information  concerning  the  warning 
triggers  and  tracks  through  the  warning  process  are  also  included,  and  therefore  this 
spreadsheet  only  covers  the  CPDS.  The  two  warning  triggers  are  listed  with  the 
categories  listed  in  Table  3.1.  The  following  columns  are  included  in  the  Warning 
Process  Spreadsheet: 

A-C)  The  date,  type  of  warning  issued  and  the  county  warned.  Note  that  for  multiple 
adjacent  warned  counties,  all  counties  are  listed  together  in  column  C. 

D)  Were  multiple  adjacent  counties  warned? 

E)  The  time  the  warning  was  issued  in  EDT  or  EST. 

F)  The  warning  number  during  the  severe  weather  episode.  Note  that  multiple 

simultaneously  warned  counties  all  get  the  same  warning  number. 

G)  The  total  number  of  cells  with  warnings  issued  during  the  episode.  In  other  words, 

multiple  warned  counties  still  are  only  treated  as  a  single  warning. 
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H)  A  detailed  account  of  the  initiator.  All  information  from  the  SWATR  is  included  in 

this  column. 

I)  The  categories  of  initiator(s)  are  listed.  Note  that  this  column  includes  all  initiators. 

J)  The  initiator-simplified  lists  only  the  first  event  causing  the  radar  operator  to  focus  his 

or  her  attention  on  the  individual  county  and  consider  issuing  a  warning  for  that 
county. 

K)  The  time  between  the  initiator  and  the  trigger. 

L-N)  These  columns  are  the  same  as  H-J  except  covering  the  trigger.  Note  that  the  last 
event  prior  to  the  issuance  of  the  warning  is  used  in  column  N. 

O)  Was  previous  verification  of  a  warning  a  trigger  for  the  issuance  of  the  warning? 

P)  Did  the  warning  have  real  time  verification? 

Q)  The  amoimt  of  time  between  the  decision  to  issue  the  warning  and  the  official 

warning  time. 

R)  Was  the  county  called  after  the  warning  was  issued? 

S)  Did  the  warning  verify  via  storm  data? 

T)  Any  additional  information  that  needs  to  be  noted  should  be  included  in  the 

discussion  section. 


89 


7.2.4  Decisions  Not  to  Issue  Warnings 

Documented  decisions  not  to  issue  warning  were  recorded  from  the  SWATR  for 
the  episodes  in  the  CPDS.  The  are  24  decisions  not  to  issue  a  warning  are  listed  in  a 
similar  manner  to  the  Warning  Process  Spreadsheet. 

A-D)  The  date,  type,  time  and  county  involved.  Similar  to  the  Warning  Process 
Spreadsheet,  if  the  operator  decides  not  to  warn  several  counties  on  the  same  cell,  there 
is  only  one  listed  decision  not  to  warn. 

E)  Was  the  decision  not  to  extend  an  existing  warning? 

F-H)  The  same  as  H-J  under  the  Warning  Process  Spreadsheet. 

I)  The  trigger(s)  for  the  decision  not  to  issue  the  warning. 

J)  The  categories  of  triggers  for  the  decisions  not  to  warn. 

K)  The  last  category  of  activity  prior  to  the  decision  not  to  issue  the  warning. 

L)  Was  the  decision  not  to  warn  correct  using  storm  data  as  the  verification  source? 

7.2.5  List  of  Misses 

The  list  of  misses  is  a  record  of  the  52  storm  data  entries  that  were  not  during 
warnings  for  the  ESDS.  This  spreadsheet  is  a  parallel  version  to  the  Master  Warning 
Spreadsheet  without  warning  specific  information. 

A-C)  The  date,  time  and  county  of  the  miss. 

D-F)  Was  the  missed  event  hail,  wind  or  a  tornado? 

G)  For  hail  misses,  was  the  hail  less  than  one  inch  in  diameter? 

H)  Was  there  a  severe  thunderstorm  or  tornado  watch? 


I)  Was  there  a  severe  call  at  the  time  of  the  storm  data  entry? 

J-K)  If  there  was  a  call  associated  with  the  storm  data  entry,  was  it  initiated  or  received? 
L)  Were  more  than  four  warnings  issued  during  the  severe  weather  episode  containing 
the  miss? 
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7.3  Spreadsheet  Excerpts 

7.3.1  Example  of  an  Event  Summary 


12  June  1995 

Severe  Thunderstorm  Watch 

Radar  Operator:  Delgado 

Radar  Recorder;  Riordan  (16:41-19:10) 

Hail,  Wind  &  Tornado  Events 

CALLS 

WARNINGS 

NOTES 

Time 

County 

r 

Initiated 

Received 

Time 

R,A 

Type 

Until 

# 

Verifies? 

(EDT) 

of  even 

B.* 

16:17 

Randolpl 

1 

Not  SVR 

A 

16:23 

Scotland 

Not  SVR 

A 

16:27 

Granville 

Not  SVR 

A 

16:35 

Scotland 

Not  SVR 

A 

16:40 

Scotland 

Not  SVR 

A 

16:42 

Vance 

Not  SVR 

A 

16:43 

Chatham 

Not  SVR 

A 

16:48 

Anson 

Not  SVR 

A 

16:48 

Stanley 

SVR 

17:15 

1 

NO 

16:48 

Stanly 

Not  SVR 

A 

16:52 

Harnett 

_ 1 

Not  SVR 

B* 

R=16:54 

16:52 

Vance 

N 

SVR 

17:30 

2 

NO 

16:55 

Granville 

Not  SVR 

A^ 

16:55 

Vance 

Not  SVR^ 

16:58 

Cumber. 

Not  SVR 

B* 

R=16:54 

17:02 

Cumber. 

Not  SVR 

B 

17:03 

Scotland 

Not  SVR 

A 

17:08 

Granville 

Not  SVR 

B* 

R=17;02 

17:08 

Hoke 

SVR 

17:45 

3 

YES 

Hail  (.75) 

17:08 

Hoke 

Not  SVR 

A 

17:12 

Orange 

Not  SVR 

B* 

R=17:16 

17:12 

Stanly 

Not  SVR 

B* 

R=17:11 

17:12 

Vance 

Not  SVR 

B* 

R=17;09 

17:14 

Johnston 

Not  SVR 

A 

17:15 

Harnett 

Not  SVR 

A 

17:16 

Wake 

Not  SVR 

A 

17:17 

Person 

Not  SVR 

B 

R=17:16 

17:18 

Hoke 

SVR 

B* 

R=17:14 

17:18 

Vance 

Not  SVR 

A 

17:20 

Chatham 

Not  SVR 

A 

17:20 

Harnett 

Not  SVR 

B* 

R=17:24 

17:22 

Scotland 

Not  SVR 

B* 

R=17:21 
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Not  SVR 
Not  SVR 


iNot  SVR 


17:23  Harnett 
17:25  Harnett  Not  SVR 

17:30  Cumber.  Not  SVR 
17:33  Cumber. 

17:34  Johnston  Not  SVFT 

17:36  ^mber.|  _ 

17:36  Johnston  _ 

17:36  Sampsor  N _ 

17:39  Cumber. 

17:40  Warren  Not  SVR 
17:41  Wake  Not  SVR 
17:41  Wake 
17:42  Wake  E 
17:43  Wake  ~r  Not  SVR 
17:50  Sampson 
17:50  Wayne  Not  SVR 
17:52  Wake  Not  SVR 
17:58  Wayne  |w 

18:00  Johnston _ 

18:00  Sampson 
18:00  Wayne  T”  Not  SVR 
18:05  Moore  Not  SVR 
18707  Wake 

18:10  Wake  Not  SVR 
18:23  Wayne  “  Not  SVR 


18:27  Halifax  Not  SVR 


18:30  Harnett 
18:33  Halifax'  _  Not  SVR 
18:36  Franklin  Not  SVR 
18:36  Wayne  Not  SVR 
18:37  Wayne  SVR 
18:40  Halifax 
18:40  Wayne  E 
18:42  Cumber.  Not  SVR 
18:43  Lee  Not  SVR 

18:44  Cumber.  Not  SVR 
18:45  Wake  Not  SVR 
18:50  Franklin  Not  SVR 

18:50  Lee  _ 

18:50  Sampson  Not  SVR 
19:03  Halifax  Not  SVR 

19:13  Halifax _ _ 

19:13  Halifax  Not  SVR 
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7.4  SAS  Data  and  Program 


Data  for  analysis  of  affects  of  population  density  and  per  capita  income  on  verification 
rate. 


County 

Warnings 

Verified 

Warnings 

Percent 

Verified 

Population 

Density 

(km'^P 

County  Area: 
(km^)* 

Per  Capita 
Income 
(dollars) 

Chatham 

13 

4 

0.31 

21.90 

1837 

19787 

Cumberland 

12 

7 

0.58 

162.30 

1706 

16403 

Durham 

11 

2 

0.18 

241.60 

772 

21547 

Edgecombe 

10 

4 

0.40 

43.20 

1312 

15432 

Franklin 

18 

7 

0.39 

28.60 

1280 

14858 

Granville 

19 

7 

0.37 

27.90 

1390 

15191 

Halifax 

8 

2 

0.25 

29.50 

1893 

14587 

Harnett 

14 

7 

0.50 

44.00 

1557  1 

14525 

Hoke 

9 

3 

0.33 

22.60 

1016 

11921 

Johnst 

22 

11 

0.50 

39.60 

2061 

17450 

Lee 

5 

2 

0.40 

62.10 

672 

19699 

Moore 

17 

10 

0.59 

32.60 

1829 

21458 

Nash 

10 

3 

0.30 

54.80 

1406 

18704 

Orange 

10 

4 

0.40 

90.60 

1039 

21945 

Person 

8 

4 

0.50 

29.70 

1047 

16849 

Scotland 

6 

2 

0.33 

40.80 

831 

15352 

Vance 

9 

4 

0.44 

59.20 

699 

Wake 

18 

13 

0.72 

196.00 

2221 

Warren 

5 

1 

0.20 

15.50 

1150 

11989 

Wayne 

17 

7 

0.41 

73.10 

1442 

15261 

Wilson 

12 

7 

0.58 

68.70 

969 

18596 
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SAS  Program 


Data  a; 

infile  ‘data.txt’  dlm=‘09’x  dsd; 

*Variable  Definitions:  warn  =  the  number  of  warning  issued  to  each  county 

*  verif  =  the  number  of  warnings  that  verify  to  each  county 

*  pop  =  the  population  density  of  the  county  per  square  km 

*  area  =  the  area  of  the  county  in  square  km 

*  income  =  the  per  caita  income  for  each  county  in  dollars 

input  warn  verif  pop  area  income; 

*Do  logistic  regression  to  see  if  the  verification  rate  is  a  function  of  the  population 

*density. 

proc  genmod; 

model  verif/wam=pop  /expected  typel  typeS  dist  =  binomial  link=logit; 

*Do  logistic  regression  to  see  if  the  verification  rate  is  a  function  of  the  per  capita 

*income 

proc  genmod; 

model  verif/wam=income  /expected  typel  typeS  dist  =  binomial  link=logit; 

*Do  Poisson  regression  to  see  if  the  number  of  warnings  issued  is  a  function  of  either  the 

*county’s  area  or  the  popultion  density. 

proc  genmod; 

make  ‘obstats’  out=pred; 

model  warn  =  area  pop  /obstats  expected  typel  type3  dist  =  poisson; 

*Do  Poisson  regression  to  see  if  the  number  of  warnings  issued  is  a  function  of  either  the 
*population  density  or  the  per  capita  income. 

Proc  genmod; 

make  ‘obstats’  out=pred; 

model  warn  =  pop  income/obstats  expected  typel  typeS  dist  =  poisson; 
run; 
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